-- Logs begin at Sun 2022-10-23 01:33:25 UTC, end at Sun 2022-10-23 02:32:41 UTC. -- Oct 23 01:57:44.336147 dv-cinder-861491-5-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Oct 23 01:57:44 2022] *** Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: nodename: dv-cinder-861491-5-iscsi Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: machine: x86_64 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: clock source: unix Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: pcre jit disabled Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: detected number of CPU cores: 4 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: current working directory: / Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: detected binary path: /usr/bin/uwsgi-core Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: your processes number limit is 47680 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: your memory page size is 4096 bytes Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: detected max file descriptor number: 2048 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: lock engine: pthread robust mutexes Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: thunder lock: enabled Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Oct 23 01:57:44.350778 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Oct 23 01:57:44.388966 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: Python main interpreter initialized at 0x55f5d018d9c0 Oct 23 01:57:44.388966 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: python threads support enabled Oct 23 01:57:44.388966 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: your server socket listen backlog is limited to 100 connections Oct 23 01:57:44.388966 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: your mercy for graceful operations on workers is 90 seconds Oct 23 01:57:44.389295 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: mapped 403077 bytes (393 KB) for 2 cores Oct 23 01:57:44.389386 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: *** Operational MODE: preforking *** Oct 23 01:57:44.389658 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: *** uWSGI is running in multiple interpreter mode *** Oct 23 01:57:44.389658 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: spawned uWSGI master process (pid: 81610) Oct 23 01:57:44.389855 dv-cinder-861491-5-iscsi systemd[1]: Started Devstack devstack@keystone.service. Oct 23 01:57:44.391240 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: spawned uWSGI worker 1 (pid: 81624, cores: 1) Oct 23 01:57:44.391240 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: spawned uWSGI worker 2 (pid: 81625, cores: 1) Oct 23 01:57:44.391240 dv-cinder-861491-5-iscsi devstack@keystone.service[81610]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Oct 23 01:57:48.685040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 01:57:48.685040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Oct 23 01:57:48.685040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 01:57:48.685040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: from cryptography.utils import int_from_bytes Oct 23 01:57:48.688663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 01:57:48.688663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Oct 23 01:57:48.689125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Oct 23 01:57:48.689125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: from cryptography.utils import int_from_bytes Oct 23 01:57:48.734152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 01:57:48.734342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 01:57:48.734560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] command line args: [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 01:57:48.734830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 01:57:48.735009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ================================================================================ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 01:57:48.735263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.735508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_dir = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.735695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.735876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] config_source = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.736055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] debug = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.736253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Oct 23 01:57:48.737569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Oct 23 01:57:48.737686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] command line args: [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Oct 23 01:57:48.737810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Oct 23 01:57:48.737936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.737997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ================================================================================ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Oct 23 01:57:48.738054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_config_append = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_dir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_dir = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] config_source = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] debug = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.738901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.739956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_config_append = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_dir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.740890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.741002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.741566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.741788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.741892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] publish_errors = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.742983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.743920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] standard_threads = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.744888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_journal = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_json = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.23/identity {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_stderr = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] publish_errors = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] use_syslog = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.745941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.746108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.746231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.746346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.746422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.746608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] standard_threads = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.746688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.746843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.746929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.747087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.747170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.747265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.747355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.747427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_journal = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.747555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.747633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_json = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.747770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_stderr = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.747910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] use_syslog = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.748106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Oct 23 01:57:48.748263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.748919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.749978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.750968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.751960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.752869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.753991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.754965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.755977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.756689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.757408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.757502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.757653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.757731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.757894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.758931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.759868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.760989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.761952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.762917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.763886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.764994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.765949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.766930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.767958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.768912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.769977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.770959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.external = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.771980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.772926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.password = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] auth.token = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.773977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.external = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.774965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.password = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] auth.token = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.775930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.776882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.777849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.778948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.779993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.780883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.781922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.782957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.783867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.784976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.785987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.786940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.787816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.788895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.789920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790730 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.790942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.791949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.792959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.793913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.794917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795072 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795315 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.795865 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.796982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.797942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.798936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.799876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.800991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.801941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.802928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.803007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.803165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.803252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.803405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.803484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.804973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.805936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.806942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.807919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.808881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.809922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.810936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.811996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.812937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.813899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.814864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.815992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.816950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.817915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.818932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.819013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.819204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.819852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.820932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.821970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.822962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81624) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 01:57:48.823587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.823815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.824062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.824308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.824546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.824791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.825026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.825275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.825512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.825757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.825990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.826231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.826486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Oct 23 01:57:48.826659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81625) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Oct 23 01:57:48.853288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.853521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.853750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.855725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.855963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.856189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.862094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.864534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.891999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.892230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.892329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.892466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.945086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.945238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.994400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.994652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.995052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:48.995291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.149947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.150082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.150164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.150299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.150390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.150542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.173956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.174949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.175741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.186948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.187420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.199780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.199966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.200938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.201063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81624) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.201145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.201319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.201540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81625) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Oct 23 01:57:49.218705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.218869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.218953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 01:57:49.219084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 01:57:49.219245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.219432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.219528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 01:57:49.219635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Oct 23 01:57:49.219718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.219916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.219997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.220185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.220458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.220703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.220783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.220973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.230718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.231722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.232014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.238543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.238998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.241548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.241917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.241999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.242206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.242286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.242512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.242659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.242802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.242886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.243999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.244282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.244364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.244559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.244641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.244832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.267146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.267544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.268607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.268898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.269019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.269310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.275991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.276268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.276351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.276550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.276630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.276844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.286611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.287126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.289243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.289536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.293090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.293391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.293686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.293768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.293978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.294957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.295136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.295215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.295410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.295683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.311002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.311288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.311373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.311571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.315185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.315457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.318409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.318689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.318772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.318991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.323241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.323351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.326468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.326553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.329685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.329767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.329995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.330992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.331196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.331278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.347746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.347867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.348044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.348168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.348318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.348405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.348589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.348674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.355557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.355727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.357117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.357289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.358685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.358844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.358979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.359132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.361745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.361840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.362041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.362123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.362432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.362535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.362742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.362825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.363022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.363112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.374681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.374788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.374972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.375935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.376016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.376224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.376313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.376511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.376591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.392418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.392523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.395356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.395447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.398299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.398520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.399840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.399998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.401454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.401728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.401816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.402016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.402099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.402305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.408366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.408717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.409840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.410210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.412857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.413862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.414098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.421582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.421878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.421960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.422163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.422247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.422572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.430468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81624) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.430751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.430876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81625) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Oct 23 01:57:49.431039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.431882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.432869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.433970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.434164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81624) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.434245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.434524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81625) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Oct 23 01:57:49.468227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f5d018d9c0 pid: 81624 (default app) Oct 23 01:57:49.468512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55f5d018d9c0 pid: 81625 (default app) Oct 23 01:57:49.471552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbc57-7d3c-43d1-a4fa-64ceb5c1d255 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:49.471843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbc57-7d3c-43d1-a4fa-64ceb5c1d255 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:49.472119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782cbc57-7d3c-43d1-a4fa-64ceb5c1d255 None None] PATH_INFO: `/v3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:49.476886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1/1] 10.222.0.23 () {50 vars in 791 bytes} [Sun Oct 23 01:57:49 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:57:56.276626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:56.277139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:56.277139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:56.281384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1/2] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:57:56 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:57:56.286507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:56.287073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:56.287073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:56.358032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: INFO dbcounter [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] Registered counter for database keystone Oct 23 01:57:56.376978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG oslo_db.sqlalchemy.engines [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81624) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 01:57:56.383862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writer thread running {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 01:57:56.565343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81624) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 01:57:56.565645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG passlib.handlers.bcrypt [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81624) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 01:57:56.623745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:57:56.625343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:57:56.751967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 2/3] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:57:56 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 01:57:57.037116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:57:57.108456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: INFO dbcounter [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] Registered counter for database keystone Oct 23 01:57:57.130403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG oslo_db.sqlalchemy.engines [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81625) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Oct 23 01:57:57.137115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writer thread running {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Oct 23 01:57:57.257120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b16ac04-3859-4da0-859c-0929b6ccfda5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:57:57.261916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:57:57.262557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:57.262838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:57.263107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:57.271149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:57:57.324801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 01:57:57.325332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12ec413-0e10-42ac-bee0-f0b0a1595e95 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:57:57.334458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 2/4] 10.222.0.23 () {58 vars in 1324 bytes} [Sun Oct 23 01:57:57 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:57:57.341563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:57:57.358999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8ff119-762f-4703-9e0b-88890c850281 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:57:57.363424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:57:57.364539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:57.364824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:57.365099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] PATH_INFO: `/v3/projects/ab10d627c6844be3824524caf5053de2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:57.368957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] RBAC: Authorizing `identity:get_project(project_id=ab10d627c6844be3824524caf5053de2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:57:57.419656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Oct 23 01:57:57.420001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3927e2f-601e-4675-82b7-c4fcef36a7dd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:57:57.424440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 3/5] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:57:57 2022] GET /identity/v3/projects/ab10d627c6844be3824524caf5053de2 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:57:59.540995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7c2b-f824-4685-a4d3-26adc3cb486f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:59.541602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7c2b-f824-4685-a4d3-26adc3cb486f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:59.541602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba7c2b-f824-4685-a4d3-26adc3cb486f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:59.545581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 3/6] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:57:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:57:59.551096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:59.551515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:59.551650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:59.572677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:57:59.574168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:57:59.582130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 4/7] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:57:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 01:57:59.861170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba7c2b-f824-4685-a4d3-26adc3cb486f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:57:59.873491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba7c2b-f824-4685-a4d3-26adc3cb486f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:57:59.877714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:57:59.878319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:59.878632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:59.878912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:59.886652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:57:59.888442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dcfbc-02c6-4bdb-bb8b-49a67e8fe207 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:57:59.897068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 4/8] 10.222.0.23 () {58 vars in 1324 bytes} [Sun Oct 23 01:57:59 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 01:57:59.903286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:57:59.912157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9af2be-2cc0-4764-a7a9-762640899841 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:57:59.917457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:57:59.918073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:57:59.918351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:57:59.918646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] PATH_INFO: `/v3/users/ef10648b88704f08acf4b18d8fe4cfac` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:57:59.919934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] RBAC: Authorizing `identity:get_user(user_id=ef10648b88704f08acf4b18d8fe4cfac)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:57:59.921737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cc6c5-7297-42bb-9e68-06bd61bb4476 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:57:59.923266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 5/9] 10.222.0.23 () {56 vars in 1198 bytes} [Sun Oct 23 01:57:59 2022] GET /identity/v3/users/ef10648b88704f08acf4b18d8fe4cfac => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:02.107168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212da75-657f-4426-869e-9c3c129778a3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.107667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212da75-657f-4426-869e-9c3c129778a3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.107667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e212da75-657f-4426-869e-9c3c129778a3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.108971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 5/10] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:02.114062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.114500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.114646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.133640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:02.134867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:02.171218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c2a4-3037-427c-9b1d-16d97b6f4594 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.171505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c2a4-3037-427c-9b1d-16d97b6f4594 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.171782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da12c2a4-3037-427c-9b1d-16d97b6f4594 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.173101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 6/11] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:02.178292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.178602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.178878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.218262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 6/12] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:02.221368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81625) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Oct 23 01:58:02.221764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG passlib.handlers.bcrypt [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81625) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Oct 23 01:58:02.271341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:02.272900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:02.369465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 7/13] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:02.490213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.514985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:02.595302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e0be2-cc39-4929-a824-8ffa1abd6762 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:02.627602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:02.628167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.628451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.628728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.636748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.640469 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:02.649876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:02.650822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:02.652077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 7/14] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:02.656612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.738142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:02.743536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174ac26a-0537-4a8e-96a2-eabab663c294 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:02.746235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:02.758300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a4dfa7-48bb-42d3-aef2-1b52b7a4aab2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:02.781010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:02.781860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.782312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.782613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.783565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:02.784989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:02.792066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:02.792675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.792957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.793232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.794577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:02.794888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:02.796273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:02.797554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 01:58:02.797554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 01:58:02.797554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [parameters: {'id': 'e49f472532844193ba0a71084b199534', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 01:58:02.797554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 01:58:02.809058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 8/15] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:02.814589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:58:02.816803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 01:58:02.817665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 01:58:02.818641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 01:58:02.819468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 01:58:02.820581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:58:02.821458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'id': 'e49f472532844193ba0a71084b199534', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 01:58:02.822254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:02.823012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 8/16] 10.222.0.23 () {60 vars in 1137 bytes} [Sun Oct 23 01:58:02 2022] POST /identity/v3/domains => generated 190 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:02.823896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.915579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3322916-ba52-4e73-ba86-04c966fecba8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:02.924452 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6b7fdc-28f0-4444-8acf-3a23f8b17a02 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:02.949529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:02.950156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.950468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.950743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] PATH_INFO: `/v3/users/ef10648b88704f08acf4b18d8fe4cfac` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.951881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] RBAC: Authorizing `identity:get_user(user_id=ef10648b88704f08acf4b18d8fe4cfac)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:02.953347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:02.954952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 9/17] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:02 2022] GET /identity/v3/users/ef10648b88704f08acf4b18d8fe4cfac => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:02.957914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:02.958806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:02.959086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:02.959391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:02.960291 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:02.976850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:02.978113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:02.979690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:58:02.995998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:02.997036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:58:02.997863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:58:02.997863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:58:02.997863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:02.997863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 9/18] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:03.003042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:03.028131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:03.063094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a80429-7cdb-4fa0-8c5a-58903fab4a88 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:03.097139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:03.097767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:03.098045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:03.098320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:03.099471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:03.100945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:03.102785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 10/19] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:03.108136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:03.109818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6072f86f-d6c7-4796-a26a-505b93d414c1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:03.143449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:03.144049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:03.144325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:03.144595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:03.145226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:03.145526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:03.146960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b127268-a0a7-439b-80c0-f4d0f931e9cb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:03.165365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 10/20] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:58:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:03.195764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:03.216632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72499512-4881-4d3a-b970-7f835bbb0242 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:03.253603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:03.254284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:03.254599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:03.254885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] PATH_INFO: `/v3/domains/default/users/ef10648b88704f08acf4b18d8fe4cfac/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:03.256915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=ef10648b88704f08acf4b18d8fe4cfac, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:03.258770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce77a41f-dcec-466a-ba72-153a90083bd8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:03.375753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 11/21] 10.222.0.23 () {58 vars in 1361 bytes} [Sun Oct 23 01:58:03 2022] PUT /identity/v3/domains/default/users/ef10648b88704f08acf4b18d8fe4cfac/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:58:05.759310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773f99-5f26-4430-b3ed-5a92dab61d21 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:05.759807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773f99-5f26-4430-b3ed-5a92dab61d21 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:05.759877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76773f99-5f26-4430-b3ed-5a92dab61d21 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:05.761512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 11/22] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:05.767110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956e7f-98dd-4872-9d0e-865d8741adf2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:05.767547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956e7f-98dd-4872-9d0e-865d8741adf2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:05.767730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2956e7f-98dd-4872-9d0e-865d8741adf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:05.795584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c2956e7f-98dd-4872-9d0e-865d8741adf2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:05.797136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c2956e7f-98dd-4872-9d0e-865d8741adf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:05.884596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 12/23] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:06.015474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b62a00-b63a-4134-b605-00ab4216885c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.016071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b62a00-b63a-4134-b605-00ab4216885c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.016554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b62a00-b63a-4134-b605-00ab4216885c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.018375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 12/24] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:06.024662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6afe0b-fe90-450f-8eac-dda6cd4949ef None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.024994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6afe0b-fe90-450f-8eac-dda6cd4949ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.025282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6afe0b-fe90-450f-8eac-dda6cd4949ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.051504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8f6afe0b-fe90-450f-8eac-dda6cd4949ef None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:06.052739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8f6afe0b-fe90-450f-8eac-dda6cd4949ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:06.130659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:06.141271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 13/25] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:06.382607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661ac8-4c9f-437f-9abb-af4f5d692a12 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.382607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661ac8-4c9f-437f-9abb-af4f5d692a12 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.382607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26661ac8-4c9f-437f-9abb-af4f5d692a12 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.384041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 13/26] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:06.390375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1f98-d171-415c-b185-a5c86d42e45d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.391029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1f98-d171-415c-b185-a5c86d42e45d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.391495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1f98-d171-415c-b185-a5c86d42e45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.402199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661ac8-4c9f-437f-9abb-af4f5d692a12 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:06.440334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-087f1f98-d171-415c-b185-a5c86d42e45d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:06.443306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-087f1f98-d171-415c-b185-a5c86d42e45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:06.548708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:06.580620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26661ac8-4c9f-437f-9abb-af4f5d692a12 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:06.586465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 14/27] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:06.586465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a887-4a31-43f3-90a3-2d0b07c35d01 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.586465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a887-4a31-43f3-90a3-2d0b07c35d01 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.586465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073a887-4a31-43f3-90a3-2d0b07c35d01 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.590604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 15/28] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:06.591360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a887-4a31-43f3-90a3-2d0b07c35d01 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:06.659191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:06.660220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.660698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.661164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.682751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:06.684603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:06.717418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073a887-4a31-43f3-90a3-2d0b07c35d01 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-101bdb0e-9efd-4147-ac09-02fa024f9e39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:58:06.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:06.721076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:58:06.722041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:58:06.722041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:58:06.722041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:06.722041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 14/29] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:06.722342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.722797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.723199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.755177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:06.760428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:06.762421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:06.763778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:06.764644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.765306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.765733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.791999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:06.797937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:06.798816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:06.799551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:06.802182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 16/30] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:06.807584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:06.929464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74116f95-2dc9-4c44-a873-6876907735db None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:06.954299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 15/31] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:06.955325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:06.977246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:06.979082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:06.979733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:06.980262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:06.981638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:06.982190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:06.984497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.011398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 17/32] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:58:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:07.013017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.026086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.085179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533b81e1-8283-45bc-9965-245745d7d04f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.098538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.162944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.163857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.166400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.167458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:07.168786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.170630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.190209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 16/33] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:07.196160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.217774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2205820-de8c-4145-a3c4-be67d5e8d489 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.278179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.278534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.278823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.279102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.279780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.281184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.301150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 18/34] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Oct 23 01:58:07 2022] POST /identity/v3/roles => generated 225 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:07.302043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.314989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841c9fc-778e-414d-acef-b9b18eb0e53e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.345259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.359316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.360225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.360508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.361223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.361785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.363428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9261222e-72d7-48fe-97c4-e1eeff0b1aa6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.447891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a987224-ed5b-4373-9be4-aef5c1305d52 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.461227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.489630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.490527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.490979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.491394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] PATH_INFO: `/v3/users/ef10648b88704f08acf4b18d8fe4cfac` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.493269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] RBAC: Authorizing `identity:get_user(user_id=ef10648b88704f08acf4b18d8fe4cfac)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.494969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.496702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 19/35] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:07 2022] GET /identity/v3/users/ef10648b88704f08acf4b18d8fe4cfac => generated 247 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:07.497595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.536899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 17/36] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:58:07 2022] POST /identity/v3/projects => generated 301 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:07.538677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354d17-e05b-44c1-acbe-6557924b001e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.538677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354d17-e05b-44c1-acbe-6557924b001e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.539007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0354d17-e05b-44c1-acbe-6557924b001e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.540361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 18/37] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:07.542099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354d17-e05b-44c1-acbe-6557924b001e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.665919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0354d17-e05b-44c1-acbe-6557924b001e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.713490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.714477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.714905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.715325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.716742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.718783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea7584c-ae4b-4eae-9c5e-e74cdab0c9f9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.720629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 19/38] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:07.721962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa7de-e92c-4ca2-bebf-7914d8f838c6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.722247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa7de-e92c-4ca2-bebf-7914d8f838c6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.722549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0faa7de-e92c-4ca2-bebf-7914d8f838c6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.726994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 20/39] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:07.726994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aee6-7802-4578-b162-b9b994f06898 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.726994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aee6-7802-4578-b162-b9b994f06898 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.726994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34aee6-7802-4578-b162-b9b994f06898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.737763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e49f7-a7f1-40cf-b0db-285d641b56a9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.746261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fb34aee6-7802-4578-b162-b9b994f06898 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:07.747836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fb34aee6-7802-4578-b162-b9b994f06898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:07.780275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.780745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.781026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.781304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.781974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.783435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:07.796418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.803892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 20/40] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Oct 23 01:58:07 2022] POST /identity/v3/roles => generated 219 bytes in 306 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:07.806158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:07.831319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 21/41] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:07.833063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.833360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.833643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.858019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:07.866903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:07.870682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:07.945195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718906f2-c051-46eb-a5c6-1dd6ece3e21c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:07.954014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 22/42] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:07.995099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:07.995945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:07.996236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:07.996517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:07.997247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.domain.id=default, user.id=ef10648b88704f08acf4b18d8fe4cfac) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:07.997565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:07.999034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:08.007074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 21/43] 10.222.0.23 () {56 vars in 1322 bytes} [Sun Oct 23 01:58:07 2022] GET /identity/v3/role_assignments?user.id=ef10648b88704f08acf4b18d8fe4cfac&scope.domain.id=default&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 530 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:08.116568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:08.226038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4154c0-95d7-4c2b-9776-b20f39a8495e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:08.229612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:08.238761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:08.265343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:08.265908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:08.266197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:08.266506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:08.267611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:08.269086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:08.270756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 23/44] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:08.276818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:08.342133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0dc6658-ae9a-426a-94f8-21ba5e947274 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:08.343943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:08.375439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065db2e-3644-433a-95e0-c36db7fc0ba1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:08.378563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:08.379175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:08.379461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:08.379742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:08.380411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:08.381781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c815028c-e55c-4793-b029-be89a1b9dc09 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:08.398072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 22/45] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Oct 23 01:58:08 2022] POST /identity/v3/roles => generated 223 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:58:08.410165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:08.410845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:08.411130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:08.411406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:08.412085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:08.413606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e32c92b-7510-4e20-a028-c630d8e7e475 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:08.442502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 24/46] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:58:08 2022] POST /identity/v3/projects => generated 312 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:58:10.610859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72728a85-3466-4744-88b7-70753a776190 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:10.611374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72728a85-3466-4744-88b7-70753a776190 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:10.611459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72728a85-3466-4744-88b7-70753a776190 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:10.612802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 23/47] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:10.618170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:10.619108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:10.619356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:10.637560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:10.638787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:10.655871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:10.706499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 25/48] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:10.986022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72728a85-3466-4744-88b7-70753a776190 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:11.085303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72728a85-3466-4744-88b7-70753a776190 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:11.118063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:11.118704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:11.118995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:11.119278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:11.120398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:11.121861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434c60f4-d5a4-49ca-b038-e8794c553c3f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:11.123445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 24/49] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:11.128941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:11.224322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bdc88a-98df-490e-af42-289a2261ddc8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:11.257271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:11.257816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:11.258954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:11.259121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:11.259710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:11.261152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55557d66-4f52-44a3-b9f2-07b130c9be73 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:11.285487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 26/50] 10.222.0.23 () {60 vars in 1140 bytes} [Sun Oct 23 01:58:11 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:13.439686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84343c-35cd-4a3e-814f-7f9e5298686b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:13.440205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84343c-35cd-4a3e-814f-7f9e5298686b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:13.440288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f84343c-35cd-4a3e-814f-7f9e5298686b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:13.441591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 25/51] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:13.447603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:13.448007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:13.448161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:13.461029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:13.467322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:13.469291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:13.534050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 27/52] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:13.810165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84343c-35cd-4a3e-814f-7f9e5298686b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:13.840434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:13.911623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f84343c-35cd-4a3e-814f-7f9e5298686b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:13.945684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:13.946309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:13.946625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:13.946907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:13.948044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:13.949499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cd8cb2-dfeb-47e0-815e-946f5dbb0433 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:13.951080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 26/53] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:13.956734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:14.054255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefe4f4-68dd-4221-92e9-380386ae6bca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:14.191270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:14.191879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:14.192166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:14.192445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:14.193103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:14.194569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f95b21-b00e-41c4-b68c-abd5c407462f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:14.206617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:14.221995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 28/54] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:13 2022] POST /identity/v3/users => generated 275 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:17.436453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6bb3b-b742-4461-890c-5e2b0427d07c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.437005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6bb3b-b742-4461-890c-5e2b0427d07c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.437005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6bb3b-b742-4461-890c-5e2b0427d07c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.438362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 27/55] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:17.443966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1d44-7f02-469f-9078-4b802377bc36 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.444410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1d44-7f02-469f-9078-4b802377bc36 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.444507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76e1d44-7f02-469f-9078-4b802377bc36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.471492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b76e1d44-7f02-469f-9078-4b802377bc36 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:17.472716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b76e1d44-7f02-469f-9078-4b802377bc36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:17.568536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 29/56] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:17.631211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71058-01ad-4ba6-8ef4-7c86a03b17e2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.631506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71058-01ad-4ba6-8ef4-7c86a03b17e2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.631791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b71058-01ad-4ba6-8ef4-7c86a03b17e2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.633179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 28/57] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:17.638677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0171-a565-432b-ade4-d88885b49b02 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.638974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0171-a565-432b-ade4-d88885b49b02 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.639256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be0171-a565-432b-ade4-d88885b49b02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.662707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c1be0171-a565-432b-ade4-d88885b49b02 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:17.663904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1be0171-a565-432b-ade4-d88885b49b02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:17.769076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 30/58] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:17.873144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0fcc-80ff-45de-bbba-130f5125886f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.873441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0fcc-80ff-45de-bbba-130f5125886f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.873723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4a0fcc-80ff-45de-bbba-130f5125886f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.875358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 29/59] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:17.880326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b067a9-032c-4908-a2af-6193088bb4e5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:17.880802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b067a9-032c-4908-a2af-6193088bb4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:17.881251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b067a9-032c-4908-a2af-6193088bb4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:17.893316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0fcc-80ff-45de-bbba-130f5125886f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:17.919039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-19b067a9-032c-4908-a2af-6193088bb4e5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:17.920238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-19b067a9-032c-4908-a2af-6193088bb4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:17.991831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.019912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 31/60] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:18.022338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0aea-3c5e-4a14-8a38-4896e0c9f811 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.022856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0aea-3c5e-4a14-8a38-4896e0c9f811 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.023490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0d0aea-3c5e-4a14-8a38-4896e0c9f811 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.026033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 32/61] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:18.032064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.032620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.033260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.039142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.047022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4a0fcc-80ff-45de-bbba-130f5125886f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.057028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:18.058958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:18.085359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.086541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.087007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.087490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.099011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:18.105142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:18.106022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:18.106776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:18.109121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 30/62] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:18.116830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.138302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 33/63] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:18.198555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.290905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd538b69-710c-4633-968f-89eeefcd67f3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.340861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.341510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.341796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.342358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.343104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:18.343677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.346077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.359149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 31/64] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.360357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.369975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a2a16c-67e3-486b-a666-918fa5082690 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.385629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.386920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.412757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.414035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.415071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.415749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.430247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:18.439723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:18.440850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:18.441791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:18.443756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 34/65] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:18.444713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.480477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e378d03-9846-4162-8d0f-a21dbb6ecdad None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.514342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.515001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.515287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.515563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.525633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:18.530940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:18.531825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:18.532553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 32/66] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:18.533174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.554532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbe224c-8069-42b0-a72d-01e209955444 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.588403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.589036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.589601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.604919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.606403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.608077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 35/67] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.608845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.611942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.632220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7096943-6310-44b4-88c1-36dbebadd189 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.639019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.665516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.666151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.666434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.666753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.676797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4c207617-d279-4b50-b771-fee56b450770 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:18.683612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:18.684766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 33/68] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:18.685514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981ffc-2a99-4fc0-8608-b4a49c67f5a0 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.686178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981ffc-2a99-4fc0-8608-b4a49c67f5a0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.686178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44981ffc-2a99-4fc0-8608-b4a49c67f5a0 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.691103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 34/69] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:18.691103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44981ffc-2a99-4fc0-8608-b4a49c67f5a0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.715012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85baf7e6-d397-44d2-9528-8daf2e495e04 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.748244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.748859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.749147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.749428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.750098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:18.750394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.751816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.759813 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 36/70] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.760396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.797102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44981ffc-2a99-4fc0-8608-b4a49c67f5a0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.830425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.830987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.831269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.831540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.832187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:18.832495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.833889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.842128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 35/71] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.842934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.845677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.854887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:18.864950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9134fb-b2c3-4e69-ab79-253398b147aa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.897948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.898605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.899450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.899729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.901022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.902493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8804656c-4669-4a45-b6cf-f69236ba8305 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.904099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 37/72] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.905271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.905560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.905836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.924410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:18.925607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:18.941322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b616a-a8d1-4c9a-a575-054c0f04b50a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:18.975483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:18.976091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:18.976372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:18.976648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:18.977471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:18.977707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:18.979123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:18.991837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 36/73] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:18.991837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:18.996845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 38/74] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:18.997685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.072327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.091425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.103765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a4587-8c0d-47fe-96c2-409930301471 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.112088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a675c422-5c54-420f-925f-a03b896d3692 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.140360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.140997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.141281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.141554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] PATH_INFO: `/v3/users/ef10648b88704f08acf4b18d8fe4cfac` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.142949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=ef10648b88704f08acf4b18d8fe4cfac)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.144403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.146004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 39/75] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/users/ef10648b88704f08acf4b18d8fe4cfac => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.146763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.152462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.153320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.153602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.153874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.155049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.156508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.158202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 37/76] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:18 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.158856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.256914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb32ae39-95f3-4b22-98e1-ce59d3c4c8d3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.266296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8781e2f1-e99d-4c80-921f-6bf07fca17d0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.293961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.294647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.294933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.295208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.296371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.297244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, user_id=758b98dda3db4b77bfb2d5ca9ed32252, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.298762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.301694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.302314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.302637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.302913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.304039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.306080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.307798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 38/77] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:19 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.308699 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 40/78] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:19 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:19.321398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.351687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.428283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9355597b-d323-43f4-9883-63c55116ec94 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.444863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c81d8c-9090-47a0-9ac8-43a2b7ce412e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.467576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5942-e38e-454c-b265-343857697889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.468989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5942-e38e-454c-b265-343857697889 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.469276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5942-e38e-454c-b265-343857697889 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.469573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354c5942-e38e-454c-b265-343857697889 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.470965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5942-e38e-454c-b265-343857697889 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.472468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354c5942-e38e-454c-b265-343857697889 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.479736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 39/79] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:19 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.479736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5942-e38e-454c-b265-343857697889 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.489309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.489960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.490238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.490547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.491672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.493880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.496209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 41/80] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:19 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.501626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.554247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.598216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.598408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354c5942-e38e-454c-b265-343857697889 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.614767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689922e-fa97-4d1a-b411-93e507c8d6a0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.637686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.638318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.638634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.638907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.640217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.641766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.643358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 40/81] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.648125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.655336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.655695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.655980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.656254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] PATH_INFO: `/v3/projects/368fd93644594953b02d621176f074c1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.657370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] RBAC: Authorizing `identity:get_project(project_id=368fd93644594953b02d621176f074c1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.658875 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.665782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 42/82] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:19 2022] GET /identity/v3/projects/368fd93644594953b02d621176f074c1 => generated 312 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:19.665782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.761033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6f4f1d-9091-4ad2-b6b4-6e87481fe64a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.777897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2213886f-d6a4-47da-a5df-04d932b9ba8a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:19.797034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.797698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.797985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.798271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/users/ef10648b88704f08acf4b18d8fe4cfac/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.800210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, user_id=ef10648b88704f08acf4b18d8fe4cfac, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.801657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.805520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.817004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 41/83] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:19 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/users/ef10648b88704f08acf4b18d8fe4cfac/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:19.817438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:19.817791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.818387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:19.818790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:19.819198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:19.824795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, user_id=758b98dda3db4b77bfb2d5ca9ed32252, role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:19.826587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:19.843645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 43/84] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:19 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:19.844859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:19.865951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:19.986605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e3688f-b984-48bd-b219-acd9dcf033f2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:20.006606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e35aa89-f538-4967-a631-b248ac3857d8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:20.028283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:20.028997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:20.029287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:20.029568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] PATH_INFO: `/v3/projects/368fd93644594953b02d621176f074c1/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:20.037031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] RBAC: Authorizing `identity:create_grant(project_id=368fd93644594953b02d621176f074c1, user_id=758b98dda3db4b77bfb2d5ca9ed32252, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:20.039051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc8c2ab-6191-4919-b4c9-b4a3a638020b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:20.069120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 44/85] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:19 2022] PUT /identity/v3/projects/368fd93644594953b02d621176f074c1/users/758b98dda3db4b77bfb2d5ca9ed32252/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:20.090474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:20.091182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:20.091473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:20.091756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:20.092445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:20.094375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3be37a-c390-4f72-854d-5b4c122f31a2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:20.136568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 42/86] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:19 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:21.764020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea853-2782-48f2-9dfa-19dd650832e4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:21.764511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea853-2782-48f2-9dfa-19dd650832e4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:21.764582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fea853-2782-48f2-9dfa-19dd650832e4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:21.766206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 45/87] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:21.771352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a32658-91ea-4b22-b7cc-0a23b01eb384 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:21.771752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a32658-91ea-4b22-b7cc-0a23b01eb384 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:21.771916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a32658-91ea-4b22-b7cc-0a23b01eb384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:21.799453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e7a32658-91ea-4b22-b7cc-0a23b01eb384 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:21.800670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e7a32658-91ea-4b22-b7cc-0a23b01eb384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:21.856017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:21.883776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 43/88] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:22.431075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fea853-2782-48f2-9dfa-19dd650832e4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:22.525931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:22.559132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fea853-2782-48f2-9dfa-19dd650832e4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:22.577993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f7880-4494-437e-96e7-e08d2a13e136 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.578287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f7880-4494-437e-96e7-e08d2a13e136 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.578597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40f7880-4494-437e-96e7-e08d2a13e136 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.579966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 44/89] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:22.584953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e249-990a-46ce-a1ff-b268a1e5fb86 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.585232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e249-990a-46ce-a1ff-b268a1e5fb86 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.585519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0642e249-990a-46ce-a1ff-b268a1e5fb86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.618264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0642e249-990a-46ce-a1ff-b268a1e5fb86 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:22.619505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0642e249-990a-46ce-a1ff-b268a1e5fb86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:22.645573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:22.646231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.646542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.646824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.665681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:22.674343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:22.675214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:22.675948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:22.676968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 46/90] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:22.688963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:22.779668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 45/91] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:22.791701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd59ee1-653d-4e51-9940-ed8621022f0f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.792294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd59ee1-653d-4e51-9940-ed8621022f0f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.792764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd59ee1-653d-4e51-9940-ed8621022f0f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.794275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 46/92] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:22.799909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.801014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.801574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.829738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:22.830982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:22.893896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5d8baa-69e4-495b-a062-f5076135c6ab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:22.978153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:22.979108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:22.979538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:22.980031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:22.980845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:22.981302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:22.981881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 47/93] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:22.987588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49981c3-07fe-41ee-a663-76406fa62d3e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:22.989580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.004355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 47/94] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:23.010391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.068060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0eafd-eb30-4743-9c6d-3d8bed8bb161 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.068884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0eafd-eb30-4743-9c6d-3d8bed8bb161 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.070214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d0eafd-eb30-4743-9c6d-3d8bed8bb161 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.071930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 48/95] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:23.077652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.078599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.079111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.087806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.123241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:23.124454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:23.213835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8792f0f2-11f9-4d8d-94eb-c12fc886a9e6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.271279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:23.271832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.272115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.272389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.275303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 49/96] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:23.276041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.277081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:23.278585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:23.280166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 48/97] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:23.288615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.468131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8482a94-939b-4206-89de-59c099e0ca98 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.473984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00fd063-f14d-4e8c-865e-bbea4e2114ac None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.487532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.524812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.531232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:23.532109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.532612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.533077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.533652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:23.534633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.535064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.535504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.556983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:23.563180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:23.564228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:23.565138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:23.567191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 50/98] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:23.573993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.691978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:23.698462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:23.723635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f252fd29-1584-475b-8c7b-e57122e4125e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.767902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:23.768542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.768824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.769100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.784512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:23.791052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:23.791885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:23.792634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:23.795042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 51/99] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:23.796543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.826187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 49/100] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:23.827079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:23.844500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.859726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:23.931732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23804e76-731a-4780-a514-499594baf5e4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.968283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15bbdc-931e-42bd-a8a6-34810417b5c6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:23.976532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:23.977491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:23.978100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:23.978547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:23.979359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:23.979906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:23.981638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:23.993718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 52/101] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:23.994933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.013630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.014302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.014627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.014901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.028100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:24.033259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:24.034174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 50/102] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:24.040484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.115040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.116388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee58929-2e54-4326-9616-348436f1eba2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.125634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.149016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9bbc4-5303-4ddd-99b3-a29745f8d1a6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.153755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.154372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.154692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.154974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.155642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:24.155952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.157350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.166071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 53/103] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.166071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.188190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.188827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.189111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.189391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.190540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.192021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cb3bb1-0099-4bbd-b76e-1c1a6e38e03e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.193664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 51/104] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.194794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32db13-680b-4175-8822-aa57c5e9b874 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.195086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32db13-680b-4175-8822-aa57c5e9b874 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.195368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a32db13-680b-4175-8822-aa57c5e9b874 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.196905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 52/105] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:24.197458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32db13-680b-4175-8822-aa57c5e9b874 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.266070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be8236d-7d19-47bd-b225-1af8bfadceb7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.300386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.300947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.301200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a32db13-680b-4175-8822-aa57c5e9b874 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.301294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.301500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.302827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.305022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860ca51c-07a0-4f0e-b278-e80e09402f08 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.306881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 54/106] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.308009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a42c-79f3-4713-a7d1-141cd4d91521 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.308301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a42c-79f3-4713-a7d1-141cd4d91521 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.309010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3744a42c-79f3-4713-a7d1-141cd4d91521 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.310402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 55/107] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:24.310972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744a42c-79f3-4713-a7d1-141cd4d91521 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.338572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52065bb-4100-47df-9613-883773636797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.339185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52065bb-4100-47df-9613-883773636797 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.339468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52065bb-4100-47df-9613-883773636797 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.339748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52065bb-4100-47df-9613-883773636797 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.340404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52065bb-4100-47df-9613-883773636797 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:24.340712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52065bb-4100-47df-9613-883773636797 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.342112 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52065bb-4100-47df-9613-883773636797 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.347324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.352631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 53/108] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.353568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52065bb-4100-47df-9613-883773636797 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.357316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.419125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3744a42c-79f3-4713-a7d1-141cd4d91521 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.453718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52065bb-4100-47df-9613-883773636797 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.454119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.454775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.455061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.455338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] PATH_INFO: `/v3/users/ef10648b88704f08acf4b18d8fe4cfac` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.456450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] RBAC: Authorizing `identity:get_user(user_id=ef10648b88704f08acf4b18d8fe4cfac)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.457918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b5795c-a1f9-47ee-91f1-e66f14189110 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.459589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 56/109] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/users/ef10648b88704f08acf4b18d8fe4cfac => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.460691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da30d-b768-478f-b26b-e2dc9c64b600 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.460980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da30d-b768-478f-b26b-e2dc9c64b600 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.461261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735da30d-b768-478f-b26b-e2dc9c64b600 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.479194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-735da30d-b768-478f-b26b-e2dc9c64b600 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:24.480378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-735da30d-b768-478f-b26b-e2dc9c64b600 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:24.490389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.491018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.491299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.491576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.492287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] RBAC: Adding query filter params (role.id=02c90212ffbd4aef9f8be3677f26ec3f, scope.project.id=73ff53385fa04ce79a87c953b158d935, user.id=758b98dda3db4b77bfb2d5ca9ed32252) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:24.492594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.495087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.500770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 54/110] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/role_assignments?user.id=758b98dda3db4b77bfb2d5ca9ed32252&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.500770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.563288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 57/111] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:24.564526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.564814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.565091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.582434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.591080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:24.592798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:24.626465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818eb951-3200-4857-bc17-59dc1ec8410b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.629066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.674736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.676640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.676930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.677206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.690207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 58/112] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:24.690579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.775403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.776895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.810970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561e917e-9e4f-446e-9132-f5f1971d1ef0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.842231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.848592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:24.849482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:24.849990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:24.850492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] PATH_INFO: `/v3/users/758b98dda3db4b77bfb2d5ca9ed32252` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:24.851838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] RBAC: Authorizing `identity:get_user(user_id=758b98dda3db4b77bfb2d5ca9ed32252)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:24.854569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:24.856504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 59/113] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/users/758b98dda3db4b77bfb2d5ca9ed32252 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:24.857504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.891125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 55/114] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:24.891821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:24.898819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:24.971578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3d03fe-70c2-444d-9281-9025272a1c6a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:24.992329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c2ad49-6e00-45da-956e-2551b1ff70ab None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.007755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.008931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.009213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.009527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.026601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.027235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.027515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.027788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.037790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:25.042868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:25.043744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:25.044471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 56/115] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:25.045259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.069015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.077347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.078926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.111811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.143693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d06a6-af69-4ca1-a1ce-6a1115a98b21 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.176640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.177237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 60/116] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:25.177335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.177517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.177797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.178074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.248085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.249549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.272664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.277185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e89ef1-c31c-4348-88a2-66ab55b0a32c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.312083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.312710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.312994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.313279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.314401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.315888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.317514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 61/117] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.323564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.323564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.347232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 57/118] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:25.348084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.419723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d5c4fe7-2aa6-462e-ad18-1ae2bb3e0b43 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.449035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cfff6a-b2e5-4bb2-90a3-6f8647210903 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.454243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.454907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.455187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.455457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.456540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.457988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.459716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 62/119] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.460314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.472393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.482992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.483612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.483899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.484178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.484838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:25.485149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.486554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.494884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 58/120] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.495658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.533479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.564123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b2e34-ca01-46a1-8c51-388924eb3fb4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.592285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373fdd-e619-4a9c-8989-4411c5ea7848 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.596594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.597224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.597506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.597781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.598924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.600376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.601994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 63/121] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.603382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.625567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.626177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.626488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.626776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.627481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] RBAC: Adding query filter params (role.id=4ba3f1bc788c4045b49a4852f9b61fc7, scope.project.id=73ff53385fa04ce79a87c953b158d935, user.id=758b98dda3db4b77bfb2d5ca9ed32252) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:25.627793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.629252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.635571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 59/122] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/role_assignments?user.id=758b98dda3db4b77bfb2d5ca9ed32252&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=4ba3f1bc788c4045b49a4852f9b61fc7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.636341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.698765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.733055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5039add-d458-42ac-ad95-c44861fcfe41 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.756102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d600cba5-2234-4fc7-b365-7bf91c6d521d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.772053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.772703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.772991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.773271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] PATH_INFO: `/v3/projects/368fd93644594953b02d621176f074c1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.774683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] RBAC: Authorizing `identity:get_project(project_id=368fd93644594953b02d621176f074c1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.776176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.777928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 64/123] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/projects/368fd93644594953b02d621176f074c1 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.783056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.788028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:25.800899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.802056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.802565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.802998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.803794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.806507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.842074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 60/124] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:58:25 2022] POST /identity/v3/projects => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:25.843021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.907592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de99e5-3c29-4900-a41d-65a11ac65db7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.944411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:25.945055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:25.945268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:25.945566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] PATH_INFO: `/v3/users/259c7ddeb5a840369e135db146ee6908` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:25.961098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] RBAC: Authorizing `identity:get_user(user_id=259c7ddeb5a840369e135db146ee6908)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:25.963260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:25.965001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 65/125] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/users/259c7ddeb5a840369e135db146ee6908 => generated 289 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:25.965803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:25.974116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836fa9d0-2e6b-41c1-8cf4-4f6ef3f56e7c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:25.976184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:26.016903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:26.017864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:26.018386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:26.018702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:26.019439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=73ff53385fa04ce79a87c953b158d935, user.id=ef10648b88704f08acf4b18d8fe4cfac) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:26.019752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:26.021358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:26.029587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 61/126] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/role_assignments?user.id=ef10648b88704f08acf4b18d8fe4cfac&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:26.030013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:26.043742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:26.092833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35b40a-2256-42f0-bb86-ee8095e6aa82 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:26.139644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:26.141017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:26.141712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:26.142310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:26.143252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] RBAC: Adding query filter params (role.id=02c90212ffbd4aef9f8be3677f26ec3f, scope.project.id=368fd93644594953b02d621176f074c1, user.id=758b98dda3db4b77bfb2d5ca9ed32252) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:26.143897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:26.146317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:26.156395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 66/127] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:25 2022] GET /identity/v3/role_assignments?user.id=758b98dda3db4b77bfb2d5ca9ed32252&scope.project.id=368fd93644594953b02d621176f074c1&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:26.171072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72b8323-b9c9-43e8-821f-4ccee5975075 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:26.214623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:26.215196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:26.215478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:26.215755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:26.216956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:26.218543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f55b47-e153-4532-a1b7-ec41a85dfc36 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:26.220605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 62/128] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:26 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:26.226862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:26.345864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05347a2-b963-4fe5-90a7-7be5212237b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:26.375464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:26.384396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:26.385061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:26.385346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:26.385640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/users/259c7ddeb5a840369e135db146ee6908/roles/7922be721103405abbfae49f51b39f15` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:26.389116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, user_id=259c7ddeb5a840369e135db146ee6908, role_id=7922be721103405abbfae49f51b39f15)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:26.389116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a9dcbc-0213-4795-81f4-99eddc01e38f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:26.403228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 67/129] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:26 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/users/259c7ddeb5a840369e135db146ee6908/roles/7922be721103405abbfae49f51b39f15 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:28.130912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09ece-af3a-4327-8ab4-b22f8e29e102 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.131417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09ece-af3a-4327-8ab4-b22f8e29e102 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.131524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d09ece-af3a-4327-8ab4-b22f8e29e102 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.132831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 63/130] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:28.138866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd6afd-12cf-4f17-b80c-a1d93439a4d5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.139273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd6afd-12cf-4f17-b80c-a1d93439a4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.139431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd6afd-12cf-4f17-b80c-a1d93439a4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.158616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f5bd6afd-12cf-4f17-b80c-a1d93439a4d5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:28.159820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f5bd6afd-12cf-4f17-b80c-a1d93439a4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:28.224394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 68/131] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:28.497044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09ece-af3a-4327-8ab4-b22f8e29e102 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:28.541646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:28.579100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f633-d728-4d33-906b-f3ae2079fc99 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.579393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f633-d728-4d33-906b-f3ae2079fc99 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.579677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e9f633-d728-4d33-906b-f3ae2079fc99 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.581031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 69/132] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:28.587143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.587446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.587727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.607073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:28.608826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:28.616392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d09ece-af3a-4327-8ab4-b22f8e29e102 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:28.652693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:28.653328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.653610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.653884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.655015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:28.656537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:28.659288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 64/133] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:28.665300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:28.680616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 70/134] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:28.776752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809a4738-55d6-4417-aa68-6aa7fe822c2f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:28.778212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:28.811824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:28.812432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:28.812715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:28.812992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:28.813642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:28.815144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:28.839010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 65/135] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:28 2022] POST /identity/v3/users => generated 283 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:58:28.958988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:29.020327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:29.076973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846003f7-0152-4ed4-b7c0-7222079dfb51 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:29.118645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:29.119525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:29.119990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:29.120265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:29.132614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:29.137949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:29.138859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:29.140168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 71/136] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:29.146308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:29.264602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c965040-84d7-44f2-9fd0-38780e5a95e1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:29.305306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:29.306339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:29.307032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:29.307549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:29.308507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:29.309012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:29.310778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:29.320383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 66/137] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:29.328153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:29.454578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5432a-13a3-452b-b3ec-8cc4eb61d9d2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:29.466623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:29.496221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:29.496862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:29.497147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:29.497424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] PATH_INFO: `/v3/users/259c7ddeb5a840369e135db146ee6908` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:29.498597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] RBAC: Authorizing `identity:get_user(user_id=259c7ddeb5a840369e135db146ee6908)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:29.500050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:29.501790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 72/138] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:29 2022] GET /identity/v3/users/259c7ddeb5a840369e135db146ee6908 => generated 289 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:29.507550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:29.534054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:29.624308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7c04e-57ef-4da4-a90f-aefbd6ba0784 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:29.660430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:29.661050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:29.661334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:29.661609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:29.735422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:29.736939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:29.758511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:29.839625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 67/139] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:29.845346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:29.953789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61654c96-54dc-47b8-ab4b-c6e6513008ac None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:29.989695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:29.990666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:29.991152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:29.991606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:29.992948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:29.994659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4810ae9-0210-430f-a6d0-be78fc923444 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:29.996640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 73/140] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:29 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:30.003553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:30.107187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642096e-0a90-4d89-9465-1baecc037c4b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:30.126523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:30.141428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:30.142033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:30.142318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:30.142627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:30.143353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] RBAC: Adding query filter params (role.id=7922be721103405abbfae49f51b39f15, scope.project.id=73ff53385fa04ce79a87c953b158d935, user.id=259c7ddeb5a840369e135db146ee6908) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:30.143666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:30.145111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28391865-b166-44f1-a86b-e3f847de3a07 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:30.151331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 68/141] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:30 2022] GET /identity/v3/role_assignments?user.id=259c7ddeb5a840369e135db146ee6908&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=7922be721103405abbfae49f51b39f15 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:31.245293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930489c-7b55-4e05-8b57-c5d05c692c78 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.245833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930489c-7b55-4e05-8b57-c5d05c692c78 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.245833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930489c-7b55-4e05-8b57-c5d05c692c78 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.247467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 74/142] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:31.252423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd57bd-6d02-44f4-bfcb-84a71d8e7506 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.252819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd57bd-6d02-44f4-bfcb-84a71d8e7506 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.252979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dd57bd-6d02-44f4-bfcb-84a71d8e7506 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.271651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-89dd57bd-6d02-44f4-bfcb-84a71d8e7506 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:31.272821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-89dd57bd-6d02-44f4-bfcb-84a71d8e7506 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:31.324729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ece0d-2d95-4391-8924-9d52aad30012 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.325016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ece0d-2d95-4391-8924-9d52aad30012 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.325294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ece0d-2d95-4391-8924-9d52aad30012 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.326841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 75/143] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:31.332276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f033-6c73-451a-9844-cc49de93a11d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.332557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f033-6c73-451a-9844-cc49de93a11d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.332834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0977f033-6c73-451a-9844-cc49de93a11d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.349746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 69/144] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:31.351978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0977f033-6c73-451a-9844-cc49de93a11d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:31.353278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0977f033-6c73-451a-9844-cc49de93a11d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:31.401087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:31.428558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 76/145] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:31.496093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2907-db52-42b8-95a8-f18b4449354c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.496363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2907-db52-42b8-95a8-f18b4449354c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.496604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa2907-db52-42b8-95a8-f18b4449354c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.498116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 70/146] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:31.503117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.503413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.503693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.522467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:31.523659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:31.588564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 77/147] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:31.621696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa2907-db52-42b8-95a8-f18b4449354c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:31.700194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:31.736000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa2907-db52-42b8-95a8-f18b4449354c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:31.751520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:31.773789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:31.774608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.775002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.775277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.787082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:31.792428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:31.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:31.794302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 71/148] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:31.799078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:31.807583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d273fa-6e05-4cf3-b648-75fe732de16f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:31.811017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:31.848085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:31.848718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.849000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.849275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.861002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:31.866034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:31.866942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:31.867864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 78/149] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:31.868522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:31.903763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cca766-9954-41e1-93ce-4461980e209c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:31.936769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:31.937379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:31.937656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:31.937940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:31.938610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:31.938915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:31.940293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:31.948739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 72/150] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:31.949518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:31.968709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6825ff-bce9-4f84-8915-73c393c68a95 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:31.983901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.004091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.004643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.004926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.005196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.006286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.007752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.009937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 79/151] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:32.010801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.040390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.054868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc96519b-7136-402e-8f1e-76e215012d05 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.088319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.088925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.089208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.089482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.090139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:32.090445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.091858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.099862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 73/152] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:32.100639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.112268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca67fba-8902-4396-bb78-23fe2aa663ff None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.147123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.147748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.148029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.148306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] PATH_INFO: `/v3/users/9a14f533213746018c32a7c81e5e86e9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.162152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] RBAC: Authorizing `identity:get_user(user_id=9a14f533213746018c32a7c81e5e86e9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.163732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.165669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 80/153] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:32 2022] GET /identity/v3/users/9a14f533213746018c32a7c81e5e86e9 => generated 283 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:32.170942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.195771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.200289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136a20e2-854b-4401-9ca1-60d88becd685 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.233581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.234184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.234500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.234778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.235425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.236863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.257683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 74/154] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:32 2022] POST /identity/v3/users => generated 297 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:32.259210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.260854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.272136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920dcdb-e3f6-4cbc-85f6-2110a0f4b9f3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.312746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.313402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.313684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.313949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] PATH_INFO: `/v3/users/9a14f533213746018c32a7c81e5e86e9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.315205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] RBAC: Authorizing `identity:get_user(user_id=9a14f533213746018c32a7c81e5e86e9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.316715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.318499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 81/155] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:32 2022] GET /identity/v3/users/9a14f533213746018c32a7c81e5e86e9 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:32.324919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.382383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332e25ca-d272-4766-8d01-ee4fa35b5d7d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.419065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.419717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.419998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.420273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.421451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.422978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.424661 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 75/156] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:32 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:32.431005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.448605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb8e4b-da8b-4186-a125-80a737771083 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.458575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.494430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.495169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.495450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.495766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.498773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.498773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.500068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 82/157] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:32 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:32.505718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:32.537710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:32.554962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9e2819-b874-4620-9797-a84ef773c332 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.594664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.595419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.595772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.596122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/9a14f533213746018c32a7c81e5e86e9/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.598129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, user_id=9a14f533213746018c32a7c81e5e86e9, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.599685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3402017f-2256-4489-be24-9b7c388b25c6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.615906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 76/158] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:32 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/9a14f533213746018c32a7c81e5e86e9/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:58:32.625839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6aad0d7-e0a0-4f33-ac86-9088e8304e34 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:32.670381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:32.671113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:32.671392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:32.671663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/9a14f533213746018c32a7c81e5e86e9/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:32.674001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, user_id=9a14f533213746018c32a7c81e5e86e9, role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:32.675610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56c71e5-e76b-4042-a62e-4f1d0415500f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:32.700894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 83/159] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:32 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/9a14f533213746018c32a7c81e5e86e9/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:58:34.586707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb068-1a9d-420b-ba95-65a2d87cad89 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.587264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb068-1a9d-420b-ba95-65a2d87cad89 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.587264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65deb068-1a9d-420b-ba95-65a2d87cad89 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.588811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 77/160] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:34.595040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84070-3ad7-4582-a3a0-75d82a9e8e55 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.595531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84070-3ad7-4582-a3a0-75d82a9e8e55 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.595601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b84070-3ad7-4582-a3a0-75d82a9e8e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.614669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f5b84070-3ad7-4582-a3a0-75d82a9e8e55 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:34.615867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f5b84070-3ad7-4582-a3a0-75d82a9e8e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:34.694506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 84/161] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:34.716575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231223f-2d10-4d39-922e-095cfe2a0206 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.716905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231223f-2d10-4d39-922e-095cfe2a0206 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.717158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231223f-2d10-4d39-922e-095cfe2a0206 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.718648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 78/162] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:34.727924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f699f-2df3-4e39-b38c-7d59b88e91d0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.728221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f699f-2df3-4e39-b38c-7d59b88e91d0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.728486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04f699f-2df3-4e39-b38c-7d59b88e91d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.747153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b04f699f-2df3-4e39-b38c-7d59b88e91d0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:34.748341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b04f699f-2df3-4e39-b38c-7d59b88e91d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:34.760029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:34.825321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 85/163] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:34.832253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdd99-fff9-4ffa-9915-fa78d4768eb7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.832790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdd99-fff9-4ffa-9915-fa78d4768eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.833329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77cdd99-fff9-4ffa-9915-fa78d4768eb7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.834907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 79/164] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:34.840803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.841033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.841305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.860197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:34.861374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:34.908310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673112b-3103-453b-8c51-18da52ddc121 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.908714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673112b-3103-453b-8c51-18da52ddc121 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.909065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1673112b-3103-453b-8c51-18da52ddc121 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.910638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 80/165] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:34.915945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:34.916230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:34.916514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:34.935281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:34.935474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 86/166] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:34.939669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:35.005969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.007979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 81/167] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:35.088210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.111758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44686838-3459-4569-b25f-2b7e84d3f43d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.126267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.147706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.148487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.148771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.149043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.150014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.160710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:35.165838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:35.167013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:35.167895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 87/168] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:35.172578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.244798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fe8928-c7f7-42d0-bc93-2e463414dd52 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.278879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7023f1-96d7-47b0-aafc-6666a2197e76 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.281317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.281965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.282244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.282545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.283637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.285090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.286660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 82/169] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:35.287406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.320634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.321840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.322121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.322390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.323086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:35.323388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.324798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.327471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.334456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 88/170] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:35.335262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.376014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.394986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1678fe5-f936-46fc-a49a-c3b366574c5b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.428147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.428774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.429058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.429330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.433662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26245af-6a53-40c3-bb2d-9427485b5076 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.439412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:35.444225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:35.445244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 83/171] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:35.447118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.472383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.472989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.473266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.473540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.474171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.475620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.497193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 89/172] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:35 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:35.498072 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.563168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad9762e-8c21-493b-a6e9-8bccb9433bfd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.563428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.601889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.603722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.603722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.603722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.615424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.617353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:35.624121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:35.625266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:35.626275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:35.627086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66dd8ae-0687-4e0c-aa0a-5537014b93b5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.627625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 84/173] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:35.632957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.665411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.667602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.668178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.668694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] PATH_INFO: `/v3/users/34473068113c4ded95652e2457f7e137` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.688569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] RBAC: Authorizing `identity:get_user(user_id=34473068113c4ded95652e2457f7e137)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.690049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.691841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 90/174] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/users/34473068113c4ded95652e2457f7e137 => generated 297 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:35.692617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.753333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc8a8ec-3120-49ed-8579-77fe713040c0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.799481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.800287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.800571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.800849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.801520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:35.801833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.803442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.811014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175e318-99fe-4099-b2d9-081b1a927d46 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.813373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 85/175] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:35.814206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.831364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.864758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.865378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.865659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.866352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.867062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:35.867370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.870316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:35.881834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 91/176] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:35.882324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:35.890477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:35.941108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823325c-7a44-4c8c-be94-c1df74ffc224 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:35.991464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:35.992099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:35.992381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:35.992652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:35.995120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:35.996589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.003061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 86/177] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:36.003061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.024606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bf78ea-97f7-40be-bf48-59aa2c07f7ee None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.068394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.069501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.070075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.070583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] PATH_INFO: `/v3/users/9a14f533213746018c32a7c81e5e86e9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.072548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] RBAC: Authorizing `identity:get_user(user_id=9a14f533213746018c32a7c81e5e86e9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.074338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.076354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 92/178] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/users/9a14f533213746018c32a7c81e5e86e9 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:36.077428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.126678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.126970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81820016-6e1a-424e-9095-e986c581d10c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.165273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.171705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.173365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.173712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.174055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] PATH_INFO: `/v3/users/9a14f533213746018c32a7c81e5e86e9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.175770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] RBAC: Authorizing `identity:get_user(user_id=9a14f533213746018c32a7c81e5e86e9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.177730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.182599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 87/179] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:35 2022] GET /identity/v3/users/9a14f533213746018c32a7c81e5e86e9 => generated 283 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:36.183576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.207927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f670b6c3-e506-4ece-856f-656cf8f018d1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.245106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.246056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.246616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.247070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/34473068113c4ded95652e2457f7e137/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.249239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, user_id=34473068113c4ded95652e2457f7e137, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.250900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.266986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 93/180] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:36 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/34473068113c4ded95652e2457f7e137/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:36.276020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.343375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24be3216-ada1-4287-a7dc-7355872ba73c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.419490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.420381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.421551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.422160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.483473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e3f0b-9687-436a-b633-5d73b69a894d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.485166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.529242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.530100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.530519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.530896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.533357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.538819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.540824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.627342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.629070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.662933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 88/181] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:36.671488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.807867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 94/182] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 532 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:36.813681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:36.815876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.874103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:36.885795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2c1e0e-6e15-4bc1-8b6e-b1ceccaf5e2b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.949751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:36.950320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:36.950777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:36.951121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:36.952348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:36.952454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ac2ee-6ce6-461b-89cc-1d6ea4b41516 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:36.953873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:36.955838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 89/183] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:36 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:36.961386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:37.047170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:37.047826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:37.048107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:37.048386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:37.055177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:37.056738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:37.058463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 95/184] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:36 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:37.064334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:37.086047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed10cd6-9e7e-4807-bd9e-f836ad7713c9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:37.159075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:37.160166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:37.161401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:37.162022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:37.164267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1, user.id=9a14f533213746018c32a7c81e5e86e9) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:37.166814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:37.168656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfe0b60-6145-42c9-beab-34b22a657ce3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:37.177415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 90/185] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:36 2022] GET /identity/v3/role_assignments?user.id=9a14f533213746018c32a7c81e5e86e9&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:37.194157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:37.241542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4fc82-7a2e-4403-b0f5-e345d58cff31 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:37.292925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:37.293934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:37.294435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:37.294993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:37.295943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] RBAC: Adding query filter params (role.id=4ba3f1bc788c4045b49a4852f9b61fc7, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1, user.id=9a14f533213746018c32a7c81e5e86e9) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:37.296451 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:37.298159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b5cedf-8907-4bd5-8678-0849943c3d89 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:37.305216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 96/186] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:37 2022] GET /identity/v3/role_assignments?user.id=9a14f533213746018c32a7c81e5e86e9&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=4ba3f1bc788c4045b49a4852f9b61fc7 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:37.989048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b2ae-2288-4ffb-9d97-89eb7178d3bb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:37.989539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b2ae-2288-4ffb-9d97-89eb7178d3bb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:37.989539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8712b2ae-2288-4ffb-9d97-89eb7178d3bb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:37.991124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 91/187] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:37.997671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d694-5d18-4bdf-8188-3473e09c1a9f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:37.998207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d694-5d18-4bdf-8188-3473e09c1a9f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:37.998207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a625d694-5d18-4bdf-8188-3473e09c1a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.018511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a625d694-5d18-4bdf-8188-3473e09c1a9f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:38.019740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a625d694-5d18-4bdf-8188-3473e09c1a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:38.090056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 97/188] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:38.119323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634700c-9da0-4a6f-b4ea-1e9936988a94 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.119590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634700c-9da0-4a6f-b4ea-1e9936988a94 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.119866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1634700c-9da0-4a6f-b4ea-1e9936988a94 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.121338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 92/189] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:38.126234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.129436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.129436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.146675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:38.147850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:38.205706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:38.215701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 98/190] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:38.364182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634700c-9da0-4a6f-b4ea-1e9936988a94 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.396197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:38.470858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1634700c-9da0-4a6f-b4ea-1e9936988a94 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:38.485761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.505536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:38.506065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.506342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.506676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.518336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:38.523201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:38.524884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:38.527131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 93/191] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:38.530901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.599014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7190115-10ff-46cb-afd1-cd2e63aff0aa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:38.625849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:38.636059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff545164-4d6e-4fc1-84fc-2dff92cbf66c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:38.636399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:38.637035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.637340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.637688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.638890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:38.640333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:38.646378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 99/192] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:38.647314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.674185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:38.674833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.675119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.675399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.676058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:38.676365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:38.677769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:38.690180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 94/193] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:38.696051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.729415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:38.775083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c68c726-19b8-4ed1-850e-9da4ea1c556f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:38.809525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab8221-082f-49cc-8dee-73f52691f284 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:38.811460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:38.812161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.812442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.812745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.813405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:38.814871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ed524-c83d-4e7f-a42e-55641b3fa81d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:38.836029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 100/194] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:38 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:38.837622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8117bc-d07e-4952-a740-a8dd630acb4c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.837907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8117bc-d07e-4952-a740-a8dd630acb4c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.838184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8117bc-d07e-4952-a740-a8dd630acb4c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.839636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 101/195] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:38.845488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.845818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.846050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.846766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:38.847448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:38.847732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:38.848004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] PATH_INFO: `/v3/users/a813afcfeb4e47b89be10f6145822681` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:38.866466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:38.867539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] RBAC: Authorizing `identity:get_user(user_id=a813afcfeb4e47b89be10f6145822681)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:38.867742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:38.869177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:38.871208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 95/196] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:38 2022] GET /identity/v3/users/a813afcfeb4e47b89be10f6145822681 => generated 297 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:38.877233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:38.891301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:38.945225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 102/197] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:38.992583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7a9c56-9721-44ee-be50-7acddfef3deb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.028737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.029370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.029651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.029925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.031072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:39.032525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:39.034166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 96/198] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:38 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:39.040433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:39.092311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:39.149006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095448a-2aad-4b68-86b2-65eaa998f68d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.186734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.187641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.188093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.188522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/a813afcfeb4e47b89be10f6145822681/roles/7922be721103405abbfae49f51b39f15` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.190493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, user_id=a813afcfeb4e47b89be10f6145822681, role_id=7922be721103405abbfae49f51b39f15)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:39.191959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:39.207230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 103/199] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:58:39 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/users/a813afcfeb4e47b89be10f6145822681/roles/7922be721103405abbfae49f51b39f15 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:39.225351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:39.330566 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:39.358518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f906189-e31a-484d-af40-19b2f6df2adf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.398941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.400514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.401021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.401496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.414138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:39.419693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:39.420571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:39.421763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 97/200] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:39.426908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:39.534863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bf885-5ce2-4d1f-9400-026e3c213717 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.547485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:39.581173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.581792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.582078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.582353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.583059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:39.583360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:39.584755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:39.593999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 104/201] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:39.599747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:39.710667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd14ae-0bca-4399-95d1-104ec38a28d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.747646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.748279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.748567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.748839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] PATH_INFO: `/v3/users/34473068113c4ded95652e2457f7e137` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.749966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] RBAC: Authorizing `identity:get_user(user_id=34473068113c4ded95652e2457f7e137)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:39.751444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:39.753324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 98/202] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:39 2022] GET /identity/v3/users/34473068113c4ded95652e2457f7e137 => generated 297 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:39.758676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:39.865792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ddc4b7-5481-40dd-a6a8-1164ef90c864 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:39.901254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:39.901872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:39.902163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:39.902464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:39.942838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:39.973946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:39.975502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:40.082358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 105/203] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:40.088527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:40.107520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:40.199704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9475f000-bc98-40d6-b5cb-021656056740 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:40.235569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:40.236198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:40.236477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:40.236747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:40.237875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:40.239434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90860e1c-0199-4a16-8a17-f3331fa1adef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:40.241060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 99/204] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:40 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:40.249626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:40.323928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:40.363497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f716b2-6b37-4144-8a2d-89c2e4c7322e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:40.399389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b76db-799a-493d-bf6d-86167309064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:40.400359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b76db-799a-493d-bf6d-86167309064e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:40.400682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b76db-799a-493d-bf6d-86167309064e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:40.400961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b76db-799a-493d-bf6d-86167309064e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:40.401693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b76db-799a-493d-bf6d-86167309064e None admin] RBAC: Adding query filter params (role.id=02c90212ffbd4aef9f8be3677f26ec3f, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1, user.id=34473068113c4ded95652e2457f7e137) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:40.402005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b76db-799a-493d-bf6d-86167309064e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:40.403454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b76db-799a-493d-bf6d-86167309064e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:40.409498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 106/205] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:40 2022] GET /identity/v3/role_assignments?user.id=34473068113c4ded95652e2457f7e137&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:41.303603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f899ae-463c-4465-85b2-da61e9c35f93 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.304120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f899ae-463c-4465-85b2-da61e9c35f93 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.304214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f899ae-463c-4465-85b2-da61e9c35f93 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.306092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 100/206] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:41.311597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14a1d5-c8f4-415d-b2cb-8d80c88f1f47 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.311997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14a1d5-c8f4-415d-b2cb-8d80c88f1f47 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.312172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c14a1d5-c8f4-415d-b2cb-8d80c88f1f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.318397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22f97-6298-4f0a-bba8-bf85dc477ad2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.318708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22f97-6298-4f0a-bba8-bf85dc477ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.318988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da22f97-6298-4f0a-bba8-bf85dc477ad2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.320341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 101/207] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:41.325510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e0f6-1104-4cd1-9924-066f5a6ad0d9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.325840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e0f6-1104-4cd1-9924-066f5a6ad0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.326159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1420e0f6-1104-4cd1-9924-066f5a6ad0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.331018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1c14a1d5-c8f4-415d-b2cb-8d80c88f1f47 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:41.332164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1c14a1d5-c8f4-415d-b2cb-8d80c88f1f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:41.344592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1420e0f6-1104-4cd1-9924-066f5a6ad0d9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:41.345771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1420e0f6-1104-4cd1-9924-066f5a6ad0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:41.406274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 107/208] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:41.420103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 102/209] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:41.499645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7950a1-5692-4168-aac3-e81793fa9701 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.500020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7950a1-5692-4168-aac3-e81793fa9701 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.500370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7950a1-5692-4168-aac3-e81793fa9701 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.504853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 108/210] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:41.507058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.507350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.507626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.517421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:41.526795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:41.527974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:41.596083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 103/211] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:41.678035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7950a1-5692-4168-aac3-e81793fa9701 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:41.693639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:41.746364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:41.786894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7950a1-5692-4168-aac3-e81793fa9701 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:41.800354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bf723b-0e57-4590-8b9f-a80a2ae76ee2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:41.848416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:41.849317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.849755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.850186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.852916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:41.853995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:41.854923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:41.855497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:41.856816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:41.858823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:41.861364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 104/212] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:41.868529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:41.869581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:41.876029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:41.877083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:41.878080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 109/213] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:41.878897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:41.879724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:41.991046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb05294-b811-42b9-8c1b-224fcbc57c51 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:41.996206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e20b1-9e98-475d-8cae-9571b7e72f0b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.022193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.033387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.033605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.033996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.034238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.034324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.034552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.034641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.034829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.035215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.036657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.045285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:42.050422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:42.051392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 110/214] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:42.056256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.061254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 105/215] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:58:41 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:58:42.072264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.145670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.210378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96ba42-b814-4d08-abc3-62f4c9b89306 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.210960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e852fe8-6712-49cf-99d8-6810dc8625ec None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.260474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.261091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.261433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.262040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.262513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.262637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:42.263140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.263275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.263411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.264289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.264391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:42.264602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.264708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.266006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.274715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 106/216] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:42.276011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 111/217] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:42.281630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.281773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.336570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.400894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce48cb9e-7a4c-4b3a-a70e-b659fa22e6fc None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.404674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5dcfc-fae2-4147-ae59-8930be075bfc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.425874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.444253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.444930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.445218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.445499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] PATH_INFO: `/v3/users/a813afcfeb4e47b89be10f6145822681` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.446382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.446675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] RBAC: Authorizing `identity:get_user(user_id=a813afcfeb4e47b89be10f6145822681)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.447082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.447363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.447636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] PATH_INFO: `/v3/users/d468b8074d834f2ea7c41b706947b7e6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.448184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.449786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 107/218] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/users/a813afcfeb4e47b89be10f6145822681 => generated 297 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:42.455387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.464291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=d468b8074d834f2ea7c41b706947b7e6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.465778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.467796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 112/219] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/users/d468b8074d834f2ea7c41b706947b7e6 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:42.473419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.589901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e411a2-cdd3-4b5b-a2a0-9ea810f38ec6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.593157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.596931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e274c-6476-4f32-b79f-6de7d7433a44 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:42.628994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.629683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.629905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.630177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.630923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] RBAC: Adding query filter params (role.id=02c90212ffbd4aef9f8be3677f26ec3f, user.id=d468b8074d834f2ea7c41b706947b7e6, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:42.631240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.632664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.638686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 113/220] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/role_assignments?user.id=d468b8074d834f2ea7c41b706947b7e6&scope.system=all&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:42.650486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:42.650486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:42.650486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:42.650486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:42.725173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:42.742947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:42.744995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:42.859985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 108/221] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:42.865723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:42.970718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff3d71d-dcb6-41cb-b829-4633642cef22 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:43.007227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:43.008284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:43.009321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:43.009796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:43.011215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:43.012885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d867169-2112-4237-ae5c-1a997e68aed1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:43.014767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 114/222] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:42 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:43.022486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:43.116389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:43.131034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b190ee13-8817-42df-84c7-5b2d650ba776 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:43.168149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:43.169104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:43.169647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:43.170120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:43.171098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] RBAC: Adding query filter params (role.id=7922be721103405abbfae49f51b39f15, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1, user.id=a813afcfeb4e47b89be10f6145822681) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:43.171614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:43.173287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23533764-02b9-43f8-ae67-f57a2137ce58 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:43.179874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 109/223] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:58:43 2022] GET /identity/v3/role_assignments?user.id=a813afcfeb4e47b89be10f6145822681&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=7922be721103405abbfae49f51b39f15 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:44.497008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc6f-c73d-423b-8d59-ea9dc3e80cca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.497520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc6f-c73d-423b-8d59-ea9dc3e80cca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.497595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ececfc6f-c73d-423b-8d59-ea9dc3e80cca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.498968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 115/224] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:44.504232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0032f4-4444-4209-bd2b-843d197c8b93 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.504646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0032f4-4444-4209-bd2b-843d197c8b93 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.504812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0032f4-4444-4209-bd2b-843d197c8b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.523550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fc0032f4-4444-4209-bd2b-843d197c8b93 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:44.523835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15116b-0fe4-44d0-a202-ac55efc83b14 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.524057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15116b-0fe4-44d0-a202-ac55efc83b14 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.524891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df15116b-0fe4-44d0-a202-ac55efc83b14 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.525510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc0032f4-4444-4209-bd2b-843d197c8b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:44.527053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 116/225] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:44.531893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5168f-79fe-42d9-9d9b-7b4ea9c8d314 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.532179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5168f-79fe-42d9-9d9b-7b4ea9c8d314 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.532454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5168f-79fe-42d9-9d9b-7b4ea9c8d314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.551004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-43d5168f-79fe-42d9-9d9b-7b4ea9c8d314 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:44.552177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-43d5168f-79fe-42d9-9d9b-7b4ea9c8d314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:44.583751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:44.598332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 110/226] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:44.626477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 117/227] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:44.819797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974365c-8a9a-4e0d-9046-a453b0e32bfe None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.820114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974365c-8a9a-4e0d-9046-a453b0e32bfe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.820302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974365c-8a9a-4e0d-9046-a453b0e32bfe None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.821947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 111/228] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:44.826845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:44.827138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:44.827415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:44.845941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:44.847143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:44.876641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974365c-8a9a-4e0d-9046-a453b0e32bfe None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:44.917303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 118/229] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:44.918143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:44.966475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:44.986605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974365c-8a9a-4e0d-9046-a453b0e32bfe None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.019342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.022647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.023265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.023550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.023828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.026974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb791e4-cf21-4a07-9e5a-7e1266f5ae8b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.035062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:45.040087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:45.041399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:45.042164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 112/230] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:45.046723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.067710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.068345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.068628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.068910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.070068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.071664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.073166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 119/231] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:45.078616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.147380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398526ae-26eb-42e3-a7ae-e582f2d0791e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.182226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.182774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12f454b-71a4-4694-b6c5-3791aadc64e4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.182903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.183132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.183405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.184115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:45.184371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.185776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.194130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 113/232] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:45.195054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.198607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.217760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.218374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.218702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.218978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] PATH_INFO: `/v3/groups` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.219866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.221330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.238417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 120/233] 10.222.0.23 () {60 vars in 1134 bytes} [Sun Oct 23 01:58:45 2022] POST /identity/v3/groups => generated 229 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:58:45.243804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.276415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.303398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ef827f-b2f7-4c51-9a06-fbea785e6055 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.348682 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.349247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.349548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.349802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.361746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:45.367256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:45.368313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:45.369154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 114/234] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:45.374673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.375897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef399422-2f14-4a74-926c-327ca288cd08 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.416716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.418170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.418426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.418848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] PATH_INFO: `/v3/users/d47716bff0324df4870fa2a507c156f9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.435388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] RBAC: Authorizing `identity:get_user(user_id=d47716bff0324df4870fa2a507c156f9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.436907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.438631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 121/235] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/users/d47716bff0324df4870fa2a507c156f9 => generated 293 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:45.444641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.449908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.498344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0662cb-1463-402b-9620-ae9915fdb54b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.532890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.533510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.534088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.534088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.534739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:45.535068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.536451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.539539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.546275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 115/236] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:45.552120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.560353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9b9374-8864-41ec-b1bd-6e791402a69b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.596860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.597800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.598362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.598664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.599435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] RBAC: Adding query filter params (role.id=7922be721103405abbfae49f51b39f15, user.id=d47716bff0324df4870fa2a507c156f9, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:45.599851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.601909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.608893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 122/237] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/role_assignments?user.id=d47716bff0324df4870fa2a507c156f9&scope.system=all&role.id=7922be721103405abbfae49f51b39f15 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:45.663508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f4006b-25aa-44a4-a139-7a83807fb3e3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.701576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:45.711548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.712888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.713176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.713455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] PATH_INFO: `/v3/users/d468b8074d834f2ea7c41b706947b7e6` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.714698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] RBAC: Authorizing `identity:get_user(user_id=d468b8074d834f2ea7c41b706947b7e6)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.716255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80bcafa-597a-474b-8c2d-0ffcc9c9b613 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.722773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 116/238] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:45 2022] GET /identity/v3/users/d468b8074d834f2ea7c41b706947b7e6 => generated 293 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:45.723958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:45.834656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8171bce-ad17-4189-aaa8-4d535184ab7b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:45.879550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60701426-8842-4958-b852-bdd991623228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:45.880206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60701426-8842-4958-b852-bdd991623228 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:45.880485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60701426-8842-4958-b852-bdd991623228 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:45.880758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60701426-8842-4958-b852-bdd991623228 None admin] PATH_INFO: `/v3/system/users/d468b8074d834f2ea7c41b706947b7e6/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:45.882158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60701426-8842-4958-b852-bdd991623228 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d468b8074d834f2ea7c41b706947b7e6, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:45.883645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60701426-8842-4958-b852-bdd991623228 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:45.894429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 123/239] 10.222.0.23 () {58 vars in 1334 bytes} [Sun Oct 23 01:58:45 2022] PUT /identity/v3/system/users/d468b8074d834f2ea7c41b706947b7e6/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:47.469185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3ce6-967a-44aa-ba20-1111f8dd51e9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:47.469964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3ce6-967a-44aa-ba20-1111f8dd51e9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:47.470038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92b3ce6-967a-44aa-ba20-1111f8dd51e9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:47.474483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 117/240] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:47.476650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657f3e-8836-40ab-a97b-158fef5e0c9c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:47.477042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657f3e-8836-40ab-a97b-158fef5e0c9c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:47.477198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f657f3e-8836-40ab-a97b-158fef5e0c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:47.496963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7f657f3e-8836-40ab-a97b-158fef5e0c9c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:47.498170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f657f3e-8836-40ab-a97b-158fef5e0c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:47.501024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:47.568637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 124/241] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:47.741617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4242-a33e-4de8-ab88-1065a6118c8a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:47.741891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4242-a33e-4de8-ab88-1065a6118c8a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:47.742077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5a4242-a33e-4de8-ab88-1065a6118c8a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:47.744127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 118/242] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:47.749120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:47.749416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:47.749689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:47.767613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:47.768779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:47.835403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 125/243] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:47.848599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a4242-a33e-4de8-ab88-1065a6118c8a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:47.959745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5a4242-a33e-4de8-ab88-1065a6118c8a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:47.995863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:47.996888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:47.997375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:47.997841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:47.999149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.000869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a7226f-ee5b-470e-8d82-0ef8bfb9c729 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.003040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 119/244] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:58:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:48.008262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.053249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e075a-10ba-453f-b883-74b78b38f4d7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.053538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e075a-10ba-453f-b883-74b78b38f4d7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.053805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e075a-10ba-453f-b883-74b78b38f4d7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.055306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 120/245] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:48.060768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.061055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.061335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.079668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:48.080839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:48.081587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.115040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b262a14e-5bee-4209-990c-65a1cd153bd1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.140110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.150615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 121/246] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:48.150879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.151434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.151533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.151850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.152100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] PATH_INFO: `/v3/groups` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.152732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.154152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.178664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 126/247] 10.222.0.23 () {60 vars in 1134 bytes} [Sun Oct 23 01:58:48 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:58:48.266067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5949f6-8561-430e-823b-ba6f09991582 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.303517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.304154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.304441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.304698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.317431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:48.323075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:48.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:48.324772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 122/248] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:48.331017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.425970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.452293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d7cb1-aa3b-44b1-b72f-20a50f6a1bd4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.483558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.495101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.496638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.497344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.497622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.497952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.499078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:48.499586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.500980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.512770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 127/249] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:48.518511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.554340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41030ce-d365-4284-882d-62b022d4ce91 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.598093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.598756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.599043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.599316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.611500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:48.616454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:48.617320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:48.618763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 123/250] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:48.625786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.648785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd91d01-9c4b-4af2-872a-36c774d4be8b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.695644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.695644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.695644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.695644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] PATH_INFO: `/v3/users/d47716bff0324df4870fa2a507c156f9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.697399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] RBAC: Authorizing `identity:get_user(user_id=d47716bff0324df4870fa2a507c156f9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.698654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.701717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 128/251] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/users/d47716bff0324df4870fa2a507c156f9 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:48.707962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.758952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf2170-6974-4b8c-9c50-17b3752d2187 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.773879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.776456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:48.802287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.802981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.803260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.803532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.804199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:48.804503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.806586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.815904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 124/252] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:48.822280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:48.825186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a744be6-de3d-4b1d-8dae-f63bdc7870b7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.872558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:48.873573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:48.874044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:48.874563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] PATH_INFO: `/v3/system/users/d47716bff0324df4870fa2a507c156f9/roles/7922be721103405abbfae49f51b39f15` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:48.876194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d47716bff0324df4870fa2a507c156f9, role_id=7922be721103405abbfae49f51b39f15)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:48.877755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:48.887479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 129/253] 10.222.0.23 () {58 vars in 1334 bytes} [Sun Oct 23 01:58:48 2022] PUT /identity/v3/system/users/d47716bff0324df4870fa2a507c156f9/roles/7922be721103405abbfae49f51b39f15 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 01:58:48.956092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bedff85-6878-46c3-b161-4b5187fdf855 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:48.999429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:49.000069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:49.000348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:49.000622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] PATH_INFO: `/v3/users/d468b8074d834f2ea7c41b706947b7e6` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:49.001779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] RBAC: Authorizing `identity:get_user(user_id=d468b8074d834f2ea7c41b706947b7e6)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:49.003303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb591e4a-e8a8-46e3-979c-489166abaf82 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:49.005228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 125/254] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:48 2022] GET /identity/v3/users/d468b8074d834f2ea7c41b706947b7e6 => generated 293 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:49.012257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:49.133158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201bc61-3203-447c-a84c-fcf2dfbc916e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:49.158050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:49.169958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:49.170612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:49.170895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:49.171172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:49.171906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] RBAC: Adding query filter params (role.id=02c90212ffbd4aef9f8be3677f26ec3f, user.id=d468b8074d834f2ea7c41b706947b7e6, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:49.172223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:49.173933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4712fd7b-d14a-438b-99a5-3061efa3bfa0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:49.188257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 130/255] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Oct 23 01:58:49 2022] GET /identity/v3/role_assignments?user.id=d468b8074d834f2ea7c41b706947b7e6&scope.system=all&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 510 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:50.634823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92513-4d4c-404f-9a10-f2b2adf0f8b2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:50.635328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92513-4d4c-404f-9a10-f2b2adf0f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:50.635431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec92513-4d4c-404f-9a10-f2b2adf0f8b2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:50.636982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 126/256] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:50.647968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7a42-676a-4a73-9fa6-6d00e37d126d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:50.648435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7a42-676a-4a73-9fa6-6d00e37d126d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:50.648509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c7a42-676a-4a73-9fa6-6d00e37d126d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:50.672506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-be4c7a42-676a-4a73-9fa6-6d00e37d126d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:50.673699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-be4c7a42-676a-4a73-9fa6-6d00e37d126d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:50.747201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 131/257] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 01:58:50.924498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf547-d623-4d59-823b-46cfd68d63b6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:50.925117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf547-d623-4d59-823b-46cfd68d63b6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:50.925593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecaf547-d623-4d59-823b-46cfd68d63b6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:50.927417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 127/258] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:50.934323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:50.934884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:50.935377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:50.955630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:50.957174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:50.970514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 132/259] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:50 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 23 01:58:51.306212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaf547-d623-4d59-823b-46cfd68d63b6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.319228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecaf547-d623-4d59-823b-46cfd68d63b6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.325918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.326810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.327306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.327911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.342951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:51.350468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:51.351332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:51.352110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:51.354913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 128/260] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:51.361144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.372199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f43e0b-79ac-4c32-814f-b521e7024103 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.381037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.381816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.382239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.382676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.383642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:51.384089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.386037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.386037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 01:58:51.386037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 01:58:51.387251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.401596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 133/261] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:51.406854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.417565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0846f8bf-1594-44cb-a92c-7ffe22240510 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.425609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.426415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.426947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.427368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.434338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.438264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.438264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 01:58:51.438264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 01:58:51.439876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.441737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 129/262] 10.222.0.23 () {56 vars in 1201 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:51.447215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.457671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd053f3-2e77-48b3-b248-346da459d153 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.463020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.463817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.464242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.464714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.465981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.468347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.470743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 134/263] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:51.476012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.487647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b289f4-90fd-4b44-8e47-39bbb8e1d6c2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.493894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.494958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.495411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.495824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/groups/6bf73995fad54b8da842bc7150ad37bf/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.497912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, group_id=6bf73995fad54b8da842bc7150ad37bf, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.499187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.499187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 01:58:51.499187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 01:58:51.501577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.520594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 130/264] 10.222.0.23 () {58 vars in 1463 bytes} [Sun Oct 23 01:58:51 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/groups/6bf73995fad54b8da842bc7150ad37bf/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:51.609782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.620835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75531adc-ecc1-4136-86e8-19e98f90f57e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.692381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.692381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.692381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.692381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.703283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:51.708976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:51.709907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:51.710744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:51.718471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 135/265] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 101 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:51.718657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.729981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd84fcce-e342-496b-a9ee-70c2d42279d0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.734834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.735519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.735851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.736131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.736827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:51.737225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.738282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.738282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 01:58:51.738282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 01:58:51.740564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c43d24-1232-41cf-a7d7-42747f5a556e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.747142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:51.751214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 131/266] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:51.757331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.769542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3726397-8806-4a24-8626-31328f81abd4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.774776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.777848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.778133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.778408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.779653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.780672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.780672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 01:58:51.780672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 01:58:51.781888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.783656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 136/267] 10.222.0.23 () {56 vars in 1201 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:51.789033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1827fe-35de-49e7-a4b1-04cb6e112845 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.789308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1827fe-35de-49e7-a4b1-04cb6e112845 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.789587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1827fe-35de-49e7-a4b1-04cb6e112845 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.790041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.791746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 132/268] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:51.797220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9d45-daa5-430e-be52-ae14a79455f9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.797958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9d45-daa5-430e-be52-ae14a79455f9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.798236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127c9d45-daa5-430e-be52-ae14a79455f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.803776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c055a-01fd-48de-8622-4a5fd8f5b18d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.816521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.818239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.818839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.820963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.825543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.828538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.833543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 137/269] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:58:51 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:51.838933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-127c9d45-daa5-430e-be52-ae14a79455f9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:51.840539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-127c9d45-daa5-430e-be52-ae14a79455f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:51.850637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:51.851763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 133/270] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Oct 23 01:58:51.863540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db695e2-e860-467c-8f1d-bdd0262ef8b1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:51.870581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:51.871602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:51.872169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:51.872658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935/groups/6bf73995fad54b8da842bc7150ad37bf/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:51.880143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ff53385fa04ce79a87c953b158d935, group_id=6bf73995fad54b8da842bc7150ad37bf, role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:51.881678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 01:58:51.881678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 01:58:51.881678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 01:58:51.884237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0a0b94-ba46-46b6-8f18-0e823d5ae234 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:51.910135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 138/271] 10.222.0.23 () {58 vars in 1463 bytes} [Sun Oct 23 01:58:51 2022] PUT /identity/v3/projects/73ff53385fa04ce79a87c953b158d935/groups/6bf73995fad54b8da842bc7150ad37bf/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:52.024309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8630-6777-42aa-88e8-644b7d787fc1 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.024972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8630-6777-42aa-88e8-644b7d787fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.025556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ae8630-6777-42aa-88e8-644b7d787fc1 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.027519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 134/272] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:52.032741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.033327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.033831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.054422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:52.056156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:52.086518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:52.143144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 139/273] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:52.439413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae8630-6777-42aa-88e8-644b7d787fc1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.480496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.537537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ae8630-6777-42aa-88e8-644b7d787fc1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.547223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.547867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.548146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.548419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.583354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:52.588438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:52.589318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:52.590079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:52.592148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 135/274] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:52.597276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.628663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4314a-2bf7-4ef4-8fde-515ad624d98f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.630308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019730ee-6c3d-4a93-aa40-3f86321194c2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.663177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.663803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.664082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.664356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.665559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:52.665861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:52.669053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:52.682480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 136/275] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:52.695571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.711129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.712690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.713249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.713719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.720907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15e29a-107c-44d8-a0e8-9cb4d1b1e7a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.736001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:52.738650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.739294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.739573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.739848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.740997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:52.742634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:52.743584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Oct 23 01:58:52.744362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:52.744942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:52.744942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 137/276] 10.222.0.23 () {56 vars in 1201 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:52.746781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 140/277] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:58:52.752408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.754563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.766026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008f41df-da2c-4147-bbcb-446246e0c2e2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.777471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.778108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.778463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.778744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.779973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:52.782063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:52.785022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 138/278] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:52.790621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:52.791055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:52.809705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b692366-7220-466a-b5d6-5ee2aef3be61 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.819654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.820308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.820584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.820856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/groups/6bf73995fad54b8da842bc7150ad37bf/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.829484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, group_id=6bf73995fad54b8da842bc7150ad37bf, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:52.832235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:52.857983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 139/279] 10.222.0.23 () {58 vars in 1463 bytes} [Sun Oct 23 01:58:52 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/groups/6bf73995fad54b8da842bc7150ad37bf/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:52.928106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4486c22c-4858-42d7-b8d1-1dc292ce4bc5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:52.968586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:52.969118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:52.969393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:52.969661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:52.970672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:52.970973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:52.972355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:52.986478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 141/280] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:52.992418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:53.114162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7a1fe5-1318-4c8f-b373-a209898fa15f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:53.116520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:53.156885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:53.157537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:53.157818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:53.158089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] PATH_INFO: `/v3/users/d47716bff0324df4870fa2a507c156f9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:53.163168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] RBAC: Authorizing `identity:get_user(user_id=d47716bff0324df4870fa2a507c156f9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:53.164770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be357d7b-bc7c-40f7-926f-41b6ddf42b38 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:53.166925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 140/281] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 01:58:52 2022] GET /identity/v3/users/d47716bff0324df4870fa2a507c156f9 => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:53.173107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:53.282116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:53.306692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e1225-63f8-4948-a22e-0914b3d85cb1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:53.346221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:53.346860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:53.347188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:53.347463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:53.348305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] RBAC: Adding query filter params (role.id=7922be721103405abbfae49f51b39f15, user.id=d47716bff0324df4870fa2a507c156f9, scope.system=all) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:53.348611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:53.350079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8363b69-5b4f-43da-b396-6b10375c8d14 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:53.363884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 142/282] 10.222.0.23 () {56 vars in 1308 bytes} [Sun Oct 23 01:58:53 2022] GET /identity/v3/role_assignments?user.id=d47716bff0324df4870fa2a507c156f9&scope.system=all&role.id=7922be721103405abbfae49f51b39f15 => generated 510 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:53.554651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c37e48-03c3-4cc6-9ece-9550c6d570cd None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:53.555320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c37e48-03c3-4cc6-9ece-9550c6d570cd None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:53.555916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c37e48-03c3-4cc6-9ece-9550c6d570cd None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:53.557691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 141/283] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:53.563789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842498-20cd-4767-9ce5-4a56cfa93ae8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:53.564394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842498-20cd-4767-9ce5-4a56cfa93ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:53.565173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b842498-20cd-4767-9ce5-4a56cfa93ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:53.595906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3b842498-20cd-4767-9ce5-4a56cfa93ae8 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:53.597497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3b842498-20cd-4767-9ce5-4a56cfa93ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:53.674320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 143/284] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:53 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 01:58:54.057500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314e2d-30c6-4ee7-a55b-2eea96dfede7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.058525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314e2d-30c6-4ee7-a55b-2eea96dfede7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.059011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd314e2d-30c6-4ee7-a55b-2eea96dfede7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.060550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 142/285] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:54.066640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.067465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.067939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.095685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:54.097222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:54.106222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 144/286] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Oct 23 01:58:54.255355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314e2d-30c6-4ee7-a55b-2eea96dfede7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.271998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd314e2d-30c6-4ee7-a55b-2eea96dfede7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.280867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.281771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.282257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.282782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.296694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:54.301870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:54.302793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:54.303554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:54.305399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 143/287] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 51 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:54.311398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.322552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca07d34-c020-4b4b-ae00-99b7992ab0b1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.330533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.331342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.331766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.332204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.333006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:54.333457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.335238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.350134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 145/288] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:54.356341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.368685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2c12d5-3c7a-4633-bdff-6236d4dfb7e9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.379095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.379940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.380363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.380798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.382107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.384776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.386524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 144/289] 10.222.0.23 () {56 vars in 1201 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:54.392116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.407996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793f2a36-c4e7-4317-acbe-2c0589d7572b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.416799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.417712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.418143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.418618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.420009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.422224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.424073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 146/290] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:54.429809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.440326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038f285e-f1ed-47a9-90e4-691baeaff416 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.447632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.448472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.448896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.449308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/groups/6bf73995fad54b8da842bc7150ad37bf/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.451658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1, group_id=6bf73995fad54b8da842bc7150ad37bf, role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.454487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.472542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 145/291] 10.222.0.23 () {58 vars in 1463 bytes} [Sun Oct 23 01:58:54 2022] PUT /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1/groups/6bf73995fad54b8da842bc7150ad37bf/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:54.715189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.728226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa772ad-a68d-4edf-be1a-957897ba8ee2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.793051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.794147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.794651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.795120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.809003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:54.814959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:54.815849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:54.816636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:54.819289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 147/292] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 105 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:54.826006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.837187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0f35e8-1fea-4a81-97b5-11272d1d82a7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.842309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.843253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.843740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.844214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.845369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:54.845881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.847772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.859517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 146/293] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:54.865747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.873750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:54.877915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f3d13-9f88-463d-a094-5183771a1dc1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.883522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.884973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.885435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.885709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] PATH_INFO: `/v3/groups/21af2eb84c304bfba70eaeb0f38b9624` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.892638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] RBAC: Authorizing `identity:get_group(group_id=21af2eb84c304bfba70eaeb0f38b9624)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.894642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.896710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 148/294] 10.222.0.23 () {56 vars in 1201 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/groups/21af2eb84c304bfba70eaeb0f38b9624 => generated 229 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:54.902587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.912981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc63c71-8b13-4e07-9526-1c978de40dc1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.918611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.919246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.919525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.919851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] PATH_INFO: `/v3/projects/ab10d627c6844be3824524caf5053de2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.922774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] RBAC: Authorizing `identity:get_project(project_id=ab10d627c6844be3824524caf5053de2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.924671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d564af-5c76-4489-a984-9208dad0201c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.926754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 147/295] 10.222.0.23 () {56 vars in 1207 bytes} [Sun Oct 23 01:58:54 2022] GET /identity/v3/projects/ab10d627c6844be3824524caf5053de2 => generated 344 bytes in 25 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:54.932583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:54.943494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956860b1-5abd-4dc8-9a03-dc7925715e5f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:54.948655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:54.949322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.949694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.950019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] PATH_INFO: `/v3/projects/ab10d627c6844be3824524caf5053de2/groups/21af2eb84c304bfba70eaeb0f38b9624/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.952120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab10d627c6844be3824524caf5053de2, group_id=21af2eb84c304bfba70eaeb0f38b9624, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:54.955013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cfc75db-310f-43d3-b293-0809408da57a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:54.980469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 149/296] 10.222.0.23 () {58 vars in 1463 bytes} [Sun Oct 23 01:58:54 2022] PUT /identity/v3/projects/ab10d627c6844be3824524caf5053de2/groups/21af2eb84c304bfba70eaeb0f38b9624/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:58:54.987308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99affa98-4dcd-4ed7-b109-edccd0062016 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:54.987611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99affa98-4dcd-4ed7-b109-edccd0062016 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:54.987874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99affa98-4dcd-4ed7-b109-edccd0062016 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:54.989662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 148/297] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:55.007385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.007690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.007939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.010890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20c2ed-e9dc-4ddd-b773-12643f233e57 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.011161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20c2ed-e9dc-4ddd-b773-12643f233e57 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.011437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa20c2ed-e9dc-4ddd-b773-12643f233e57 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.012875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 149/298] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:55.025658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.025998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.026202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.054344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:55.055717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:55.061902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:55.063187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:55.199126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 150/299] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:58:55.199814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 150/300] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:55.487053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:55.552483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:55.599634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3239b-13a7-4205-a602-5e73ffd00a84 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:55.645139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:55.645778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.646055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.646328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.661693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:55.667371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:55.669204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:55.670935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 151/301] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:55.674374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:55.705017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:55.857812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11208a59-61e2-40d1-bb39-dbd70139232a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:55.886827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:55.896898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f793612d-4914-4f4e-b04f-70c7a3a8dfc7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:55.925782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:55.926403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.926708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.926982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.927648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:55.927948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:55.929322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a03df-beae-4b4c-ba6f-891a9a84cba4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:55.938572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:55.954727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 151/302] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:55.956488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b4d4-5cbb-4677-a2ba-e355d532e56e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.956783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b4d4-5cbb-4677-a2ba-e355d532e56e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.957061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3b4d4-5cbb-4677-a2ba-e355d532e56e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.958389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 152/303] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:58:55.963599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b4d4-5cbb-4677-a2ba-e355d532e56e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:55.967773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:55.968682 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.969026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.969371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:55.981825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c320dfe8-fb53-4891-ab1e-4cfb5e0d62e4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:55.987856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:55.988730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:58:55.989523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 152/304] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:55.991087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:55.991465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:55.991846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.011252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:56.012588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:56.092659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a3b4d4-5cbb-4677-a2ba-e355d532e56e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.103396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 153/305] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:56.104504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.145815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.146424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.146726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.146999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.236877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.270669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:56.272146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:56.288593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a95c04e-8e55-43ea-abc2-bf748774997b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.304086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.344100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.344728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.345269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.345544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.346246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:56.346588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:56.348023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:56.364870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 154/306] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:56.371119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.405840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 153/307] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:56.411198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.531135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6742b94c-e620-4317-b0dc-b5b7c3077ae1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.564306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.573972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.575026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.575625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.576095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.607877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384d99fa-409d-48bc-8370-28228128b713 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.628925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.666520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.667099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.667379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.667653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.668262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:56.668795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:56.670173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:56.670278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:56.672065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 154/308] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:56.672856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.786551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717343f5-363f-4f80-a461-7851668a7385 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.795881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 155/309] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 23 01:58:56.796630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.824034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.824753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.825039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.825415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.837351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:56.842435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:56.843320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:56.844085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 155/310] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:56.844736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.850747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.879281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:56.912728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc4ffa7-70cc-4622-8b7c-9af6fbff8ee7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.951850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.952632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.952749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.953023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:56.954129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:56.955728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:56.957459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 156/311] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:56.958294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:56.959625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efb9b91-bf9a-44f4-aaa9-fccc544e6e5b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:56.997706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:56.998842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:56.999349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:56.999829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] PATH_INFO: `/v3/projects/73ff53385fa04ce79a87c953b158d935` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.001193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] RBAC: Authorizing `identity:get_project(project_id=73ff53385fa04ce79a87c953b158d935)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.002861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.004816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 156/312] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/projects/73ff53385fa04ce79a87c953b158d935 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.005809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.086170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411936b6-ca62-450a-b754-c1104f6bb017 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.097427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.126895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.127557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.127803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.128085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.128771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] RBAC: Adding query filter params (name=member) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:57.129066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.131975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.140405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b786d3f-10be-4b49-be16-5c8f8425b3ee None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.141592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 157/313] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:58:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.146350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.149417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.182677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.183320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.183605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.183886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.184620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] RBAC: Adding query filter params (group.id=6bf73995fad54b8da842bc7150ad37bf, role.id=4ba3f1bc788c4045b49a4852f9b61fc7, scope.project.id=73ff53385fa04ce79a87c953b158d935) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:57.184931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.186392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.201588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 157/314] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/role_assignments?group.id=6bf73995fad54b8da842bc7150ad37bf&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=4ba3f1bc788c4045b49a4852f9b61fc7 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.202880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.277523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370f0a1-39c3-4025-89b0-17ff98060629 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.332801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.333453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.333732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.334002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.335253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.336968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1208c4-917d-4f9b-b238-5e4abbce737c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.340268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 158/315] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.349695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43443-e76e-410e-9908-670dc8560525 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.350008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43443-e76e-410e-9908-670dc8560525 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.350241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba43443-e76e-410e-9908-670dc8560525 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.351891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 159/316] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:57 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:57.353667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba43443-e76e-410e-9908-670dc8560525 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.356639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb1f4de-d970-4663-b2dd-13dcfb577118 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.399566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.400700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.401182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.401663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.406805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.441580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.481605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba43443-e76e-410e-9908-670dc8560525 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.487772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.489319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.532345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.532960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.533240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.533509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.534236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] RBAC: Adding query filter params (group.id=6bf73995fad54b8da842bc7150ad37bf, role.id=02c90212ffbd4aef9f8be3677f26ec3f, scope.project.id=73ff53385fa04ce79a87c953b158d935) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:57.534590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.536036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acf505b-859a-4fab-8469-7f7a42d89c38 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.543394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 160/317] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/role_assignments?group.id=6bf73995fad54b8da842bc7150ad37bf&scope.project.id=73ff53385fa04ce79a87c953b158d935&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.544600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.544832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.545136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.572685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:57.574835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:57.631775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 158/318] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:57.639596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.682761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 161/319] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:57.704572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.796971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d94426-0389-4189-800e-55816ffd3bc2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:57.834346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:57.835024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.835306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.835581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.836753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:57.838273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e478a0-7e78-4d8b-a4d7-a47223f5fa48 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:57.839915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 159/320] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:57.845862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:57.870583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d8c-1b25-4755-8835-9c165f21f08d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.870857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d8c-1b25-4755-8835-9c165f21f08d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.871136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a26d8c-1b25-4755-8835-9c165f21f08d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.872794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 160/321] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:58:57.879743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:57.880031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:57.880314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:57.912889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:58:57.914127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:58:57.922317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:57.975123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bf3769-c892-4b9d-8ef1-2ed908a38e8e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.009726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 161/322] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:58:58.010827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.018688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.019323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.019600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.019868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.021383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.022858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.025095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 162/323] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Oct 23 01:58:57 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:58.030985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.057919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.132736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475be948-4ce5-4bb0-b22d-3a48a2f23889 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.145564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5dcee-d4f1-44d7-b33b-c44ff5e6b615 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.178808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.179440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.179718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.180015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.182268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.189816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.190467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.190759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.191035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.191763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] RBAC: Adding query filter params (group.id=6bf73995fad54b8da842bc7150ad37bf, role.id=02c90212ffbd4aef9f8be3677f26ec3f, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:58.192284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.193749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.199337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:58.200338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 163/324] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/role_assignments?group.id=6bf73995fad54b8da842bc7150ad37bf&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=02c90212ffbd4aef9f8be3677f26ec3f => generated 611 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:58.205364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:58.206364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:58.207203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 162/325] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:58.217492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.293514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.335146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de68fa93-f146-44cc-a920-45dd584e3d78 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.388560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.389545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.390020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.390488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.392248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:58.392832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.394401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.405833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 164/326] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:58.411585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.425423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.432229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d7f64f-0b98-4cad-b711-63e43b5e8973 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.476006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.477540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.478585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.479004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.485797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.493428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:58:58.499906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:58:58.500963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:58:58.501942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 163/327] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:58:58.510098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.533975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455f0e58-1947-4b63-bb51-c877e88391af None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.576631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.577249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.577528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.577801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.628097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730ffc9d-a511-47da-a5d0-751899f1b335 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.661502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.662157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.670364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.671062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.671337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.671608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.672258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:58.672555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.673929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.685800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 164/328] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:58:58.691217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.712250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.738289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.789611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 165/329] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 23 01:58:58.796265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.813846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fed5f2-5d30-44d1-b8e8-c6aaa3002c0c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.857103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.858205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.858205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.858394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.909688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d2072f-ab1d-4dc5-b3c0-cf3b3657de7f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:58.941245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.942231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.949891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:58.950632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:58.950917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:58.951193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] PATH_INFO: `/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:58.952448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] RBAC: Authorizing `identity:get_project(project_id=5e0c3d32fe5446fbaf8412799dfbd0d1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:58.953794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:58.955633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 166/330] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/projects/5e0c3d32fe5446fbaf8412799dfbd0d1 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:58.960146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:58.961076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:58.998675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:59.059097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 165/331] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:58:59.065653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:59.088389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255222ff-2081-49a4-9e52-0f89732b0f1c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:59.122498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:59.123124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:59.123410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:59.123677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] PATH_INFO: `/v3/groups/6bf73995fad54b8da842bc7150ad37bf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:59.124769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] RBAC: Authorizing `identity:get_group(group_id=6bf73995fad54b8da842bc7150ad37bf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:59.126202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:59.127803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 167/332] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Oct 23 01:58:58 2022] GET /identity/v3/groups/6bf73995fad54b8da842bc7150ad37bf => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:59.133485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:59.166496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eee2e7-f5fc-48c8-a1fa-1a6286e943a1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:59.182885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:59.200522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:59.201157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:59.201435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:59.201708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] PATH_INFO: `/v3/projects/ab10d627c6844be3824524caf5053de2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:59.202840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] RBAC: Authorizing `identity:get_project(project_id=ab10d627c6844be3824524caf5053de2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:59.204271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:59.205906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 166/333] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 01:58:59 2022] GET /identity/v3/projects/ab10d627c6844be3824524caf5053de2 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:59.211239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:59.228546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:58:59.235493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d95723-0961-445b-9e3f-a562d65c5485 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:59.268752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:59.269347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:59.269629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:59.269903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:59.270654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] RBAC: Adding query filter params (group.id=6bf73995fad54b8da842bc7150ad37bf, role.id=4ba3f1bc788c4045b49a4852f9b61fc7, scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:59.270964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:59.272387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:59.278664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 168/334] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Oct 23 01:58:59 2022] GET /identity/v3/role_assignments?group.id=6bf73995fad54b8da842bc7150ad37bf&scope.project.id=5e0c3d32fe5446fbaf8412799dfbd0d1&role.id=4ba3f1bc788c4045b49a4852f9b61fc7 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:59.317255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1b5ecf-7aa9-4128-8834-759bb234c03f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:59.355533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:59.356312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:59.356596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:59.356872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] PATH_INFO: `/v3/groups/21af2eb84c304bfba70eaeb0f38b9624` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:59.358077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] RBAC: Authorizing `identity:get_group(group_id=21af2eb84c304bfba70eaeb0f38b9624)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:59.359599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9eaee7-3485-40aa-826c-5172b5eb6c9d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:59.361453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 167/335] 10.222.0.23 () {56 vars in 1180 bytes} [Sun Oct 23 01:58:59 2022] GET /identity/v3/groups/21af2eb84c304bfba70eaeb0f38b9624 => generated 229 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:58:59.367268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:58:59.473927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af3a364-8eaa-47d4-92dd-af3a04a3bd0a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:58:59.510849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:58:59.511460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:58:59.511750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:58:59.512016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:58:59.512745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] RBAC: Adding query filter params (group.id=21af2eb84c304bfba70eaeb0f38b9624, role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=ab10d627c6844be3824524caf5053de2) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:58:59.513052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:58:59.514503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43927b3a-a479-4218-a484-5487707fb5cb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:58:59.520840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 169/336] 10.222.0.23 () {56 vars in 1376 bytes} [Sun Oct 23 01:58:59 2022] GET /identity/v3/role_assignments?group.id=21af2eb84c304bfba70eaeb0f38b9624&scope.project.id=ab10d627c6844be3824524caf5053de2&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:02.169143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81675-f99d-44fd-a002-3925ff2732f9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.169711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81675-f99d-44fd-a002-3925ff2732f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.169711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee81675-f99d-44fd-a002-3925ff2732f9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.171278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 168/337] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:02.176966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8412c96-d105-4c67-90a8-bb6a72c593db None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.177756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8412c96-d105-4c67-90a8-bb6a72c593db None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.178218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8412c96-d105-4c67-90a8-bb6a72c593db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.203874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a8412c96-d105-4c67-90a8-bb6a72c593db None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:02.205072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a8412c96-d105-4c67-90a8-bb6a72c593db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:02.233313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:02.308562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 170/338] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:02.516321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca788-c427-4f0c-a97a-262dfb632d80 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.516830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca788-c427-4f0c-a97a-262dfb632d80 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.517262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ca788-c427-4f0c-a97a-262dfb632d80 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.519011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 169/339] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:02.525140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.525739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.526203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.551923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:02.553356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:02.634864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd44935-0d48-478b-84a3-3553a0708459 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.635401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd44935-0d48-478b-84a3-3553a0708459 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.635833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd44935-0d48-478b-84a3-3553a0708459 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.637490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 170/340] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:02.643366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590acb2-522a-497b-97f1-f49494bacffb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.643911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590acb2-522a-497b-97f1-f49494bacffb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.644402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a590acb2-522a-497b-97f1-f49494bacffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.646387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 171/341] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:02.679851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a590acb2-522a-497b-97f1-f49494bacffb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:02.681570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a590acb2-522a-497b-97f1-f49494bacffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:02.718249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:02.770192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:02.814331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 171/342] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:02.861577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:02.885174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab955f0-b6e0-473f-a301-2351e6857a31 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:02.923823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e77ef-4bbe-46fb-804a-f4a2affcb2fc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.924467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e77ef-4bbe-46fb-804a-f4a2affcb2fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.924962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48e77ef-4bbe-46fb-804a-f4a2affcb2fc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.926503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 172/343] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:02.932516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.933059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.933566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.935757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:02.936390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:02.936667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:02.936936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:02.947630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:02.949543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:02.965790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:02.966945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:02.967806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:02.967806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:02.967806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:02.969422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 172/344] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:02.974061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:02.974061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:02.975780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.064077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 173/345] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:03.064957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.107037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d7fd3d-284b-4faf-8357-96ecfd393bba None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.172242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.172864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.173144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.173446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.174093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:03.174397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.175811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.177271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.192141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 173/346] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:03.195339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.219127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6640615-f516-4c7f-83ae-019831dc9b6b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.235255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.260483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.261597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.262131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.262660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.274041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.276051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:03.292788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:03.293727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:03.294630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:03.294630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.294630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:03.296457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 174/347] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:03.297491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.308517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4737589e-289c-4315-8f7a-ea50dbe0a6bd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.347351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.348014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.348293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.348592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.360266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.361856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:03.377880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:03.378797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:03.379621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:03.379621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.379621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:03.379621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 174/348] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:03.385063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.408026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923e72c2-5636-4238-bf39-70613cf07ac7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.432048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.443609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.444226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.444509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.444787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.445428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.447034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.474526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 175/349] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:03 2022] POST /identity/v3/users => generated 246 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:03.474526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.497975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b459e-01bb-4c80-adaa-0e0155bf9bb4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.506054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.548608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.549368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.549716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.550321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.551089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:03.551928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.553987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.572626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 175/350] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:03.573528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.610216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ce4f8d-b67f-48df-a1c7-d93c6901489a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.654088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.655288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.655924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.656318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.669384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.671415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:03.691252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:03.692368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:03.693444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:03.693444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:03.693444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:03.694571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 176/351] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:03.695315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.709938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56e66ed-bf12-4424-99d8-83b06555acfa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.718905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.753297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.753601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.753884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.754149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.754838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:03.755143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.756530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.768584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 176/352] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:03.769406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.779513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:03.815985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa09554-8afb-44b2-8989-2701f12de6d5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.854134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.854812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.855130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.855439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.856145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.857605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f298a2f-da25-4f0e-820e-b9bd753793ed None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.877611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe84ac5-6c50-4cac-850f-765e8169f9be None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:03.880983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 177/353] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:03 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:03.887024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8dbc-9ee7-4bc8-8a7e-11c738d06144 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.887024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8dbc-9ee7-4bc8-8a7e-11c738d06144 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.887024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bc8dbc-9ee7-4bc8-8a7e-11c738d06144 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.887024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 178/354] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:03.887024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc8dbc-9ee7-4bc8-8a7e-11c738d06144 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:03.922251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:03.922931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.923215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.923493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.924148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:03.924511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:03.925915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86705-bd46-423f-8402-238e7f7e4a81 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:03.940510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 177/355] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:03.941842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:03.942139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:03.942417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:03.961459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:03.963040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:03.987625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:04.019930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bc8dbc-9ee7-4bc8-8a7e-11c738d06144 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:04.047667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 178/356] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:04.048724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:04.058855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:04.067070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:04.067978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:04.068373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:04.068756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:04.069642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:04.072313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:04.107522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 179/357] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:03 2022] POST /identity/v3/users => generated 248 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:04.208660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428142e1-cb59-4d51-bd7e-213a088a7cdb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:04.252126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:04.253098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:04.253557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:04.253975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:04.254837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:04.256485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:04.290621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 179/358] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:04 2022] POST /identity/v3/users => generated 249 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:04.372362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:04.491776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db65af2c-e227-49fb-964e-50e86b43d5f3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:04.531983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:04.532860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:04.533289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:04.533700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:04.534616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:04.536117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:04.547683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 01:59:04.547683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 01:59:04.547683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [parameters: {'id': '7d53a65444fa40b08021a263528efdcf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 01:59:04.547683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:04.555998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return ref.to_dict() Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:04.557609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 01:59:04.558868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 01:59:04.559827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:04.560769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:04.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'id': '7d53a65444fa40b08021a263528efdcf', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:04.562288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Oct 23 01:59:04.563169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:04.565985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 180/359] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Oct 23 01:59:04 2022] POST /identity/v3/roles => generated 144 bytes in 194 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:04.571696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:04.590733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:04.707718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77722c5e-f998-4bda-b5cc-849d4b429e32 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:04.759332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:04.760451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:04.760733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:04.761006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:04.779659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-12d2ef03-f66f-4f99-8290-7b2156a2d502 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:04.785040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:04.785913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:04.786719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 180/360] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:04.795554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:04.870861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:04.921185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4611097d-8500-4f42-a23b-2993846f13b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:04.976913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:04.978768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:04.979065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:04.979342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:04.980009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:04.980312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:04.981684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac3803-d5ad-426d-af5a-bd893b4a8a3d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:04.990355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 181/361] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:05.897681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3cd1d-f931-48ae-b03c-09d3d8b52565 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:05.898204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3cd1d-f931-48ae-b03c-09d3d8b52565 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:05.898204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb3cd1d-f931-48ae-b03c-09d3d8b52565 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:05.900462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 181/362] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:05.905861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:05.906257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:05.906419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:05.931896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:05.933101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:06.016602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 182/363] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:06.369967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3cd1d-f931-48ae-b03c-09d3d8b52565 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:06.469420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:06.522575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb3cd1d-f931-48ae-b03c-09d3d8b52565 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:06.573862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:06.574885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.575373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.575846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.592813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-73719da0-27e9-46d6-8127-122a04d59387 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:06.599081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:06.600028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:06.600805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:06.603304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 182/364] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:06.615630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:06.674158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:06.752986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1a5df-dfd7-4e05-9efb-916fd372daf4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:06.757081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37899-9df6-4636-a69d-b5146b1f25cc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.757649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37899-9df6-4636-a69d-b5146b1f25cc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.758125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c37899-9df6-4636-a69d-b5146b1f25cc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.760426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 183/365] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:06.766461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faa11f-63d5-4af5-8782-4f39ad278564 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.767064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faa11f-63d5-4af5-8782-4f39ad278564 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.767538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7faa11f-63d5-4af5-8782-4f39ad278564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.799121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b7faa11f-63d5-4af5-8782-4f39ad278564 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:06.800442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:06.800593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b7faa11f-63d5-4af5-8782-4f39ad278564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:06.801068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.801414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.801691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.802367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:06.802707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:06.804091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:06.813737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 183/366] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:06.819357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:06.926578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 184/367] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:06.926578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac4-33e0-498f-8d85-be5a07244f04 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.926578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac4-33e0-498f-8d85-be5a07244f04 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.926578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1dac4-33e0-498f-8d85-be5a07244f04 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.926578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 185/368] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:06.942500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:06.942844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:06.943062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:06.968652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:06.969832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:06.977174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590ca1e-d818-42d4-a416-bab7a22deb48 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:06.981786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.042030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.042750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.043029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.043332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.061366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.062842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.067534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.087831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 186/369] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:07.090471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:07.091336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:07.092188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:07.092188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:07.092188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:07.092188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 184/370] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:07.104436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.229822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bda549b-967c-40c2-8f14-c66e6afec366 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.264742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.265353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.265627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.265948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.266612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:07.266961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.268319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9fbfed0-1103-4442-a4d1-603195cfbaa6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.280855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 187/371] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:07.287532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.292284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bac7e-13b7-42fa-9409-ceb57445725a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.292567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bac7e-13b7-42fa-9409-ceb57445725a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.292864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34bac7e-13b7-42fa-9409-ceb57445725a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.294320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 188/372] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:07.307419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.308610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.309269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.342201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:07.344424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:07.410122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.425974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7036d8-ce88-426e-80e4-c442c9c4bcac None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.439091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 189/373] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:07.444133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.471989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.472884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.473312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.473732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.484094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.485747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:07.503192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:07.504368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:07.505587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:07.508162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 185/374] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:07.509174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.529206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.564872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5cff6-228d-44d5-b7d0-c40e45e0f6a7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.609675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.610423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.610726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.610994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.625935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:07.631375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbd9aa-1bb1-4342-bac1-21c4a053c629 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:07.631899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:07.632797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:07.634212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 190/375] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:07.638569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.679893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.682052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.682340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.682643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.697184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.699254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:07.706278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:07.707185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:07.707987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:07.713295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 186/376] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:07.714369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.766201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0866364c-68f2-47c8-8e9b-6c0642024923 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.798587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.802880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.803616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.803847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.804120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.804850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:07.805158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.807276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.829379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48eea14-04e7-4708-9d9e-f8475404e0cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.853084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 191/377] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:07.853084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.877927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.878694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.879039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.879324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.880017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:07.880322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.881717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.890909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 187/378] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:07.895934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:07.956803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:07.959784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddaf3ac-ed10-40f7-b303-09a6cf801e0a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:07.994456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:07.995076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:07.995362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:07.995641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:07.996299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:07.996599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:07.997988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:07.998354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cea3d-7379-4b09-be9b-b2f089a97aea None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.005668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 192/379] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:08.006467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.034001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.034650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.035001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.035476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.040413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.051697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:08.057584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.058520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:08.059321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 188/380] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.059956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.117621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323cea7a-6c38-440e-a7c4-58f9d78f0f90 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.150210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.150857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.151133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.151402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.157932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0058b3df-761e-4b59-95aa-5896e6df349e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.161398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.162890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-cee65710-04a2-435a-9507-64a332b8180d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:08.177376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.178305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:08.179794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:08.179794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.179794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:08.179794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 193/381] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.179794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d64ba-bbd3-4399-85d3-7d93f1b8c06e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.180087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d64ba-bbd3-4399-85d3-7d93f1b8c06e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.180150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d64ba-bbd3-4399-85d3-7d93f1b8c06e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.181907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 194/382] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:08.182432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d64ba-bbd3-4399-85d3-7d93f1b8c06e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.195975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.196810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.197276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.197558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.197969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.208236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.209683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:08.224648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.225617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:08.226462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:08.226462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.226462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:08.226462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 189/383] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.226678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.279348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.290330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d64ba-bbd3-4399-85d3-7d93f1b8c06e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.323999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.324610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.324894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.325169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.331164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080868e-3dbc-4f4a-b385-be7b9d340cbc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.335991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.337527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:08.351266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.352859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:08.353936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:08.353936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.353936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:08.353936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 195/384] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.353936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.367526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.368777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.368777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.369272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.369927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:08.370229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.371657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b0cb1-f778-4f4b-a40c-796f0f630ba1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.379832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 190/385] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:08.381122 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.381415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.381699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.398781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:08.400057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:08.426582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.461496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bbc0b4-b5ae-495b-bdd6-f2d73d561d68 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.467962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 191/386] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:08.468775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.498199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.498843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.499129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.499409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.500041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:08.500351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.501721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.515162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 196/387] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:08.516420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.520548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.577623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef6a013-9761-456f-9dd0-0ee2f16f92b6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.613704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.614299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.614615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.614889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.615519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:08.615820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.617206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.628062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4bd19a-1399-428a-a7f5-0f2287159a9c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.635184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 192/388] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:08.635184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.661443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.671656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.672408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.672757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.673149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.673852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:08.674335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.675827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.694369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 197/389] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:08.694369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.748245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.751586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63786669-ccbe-4278-8026-d5c3da0dc844 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.786770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.787398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.787682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.787958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.794974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab205221-ddd9-44d5-a489-a33ab59ec0ef None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.799044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.800489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:08.814281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.815374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:08.816233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:08.816233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:08.816233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:08.816233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 193/390] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.816233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.830408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.830986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.831261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.831529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.840137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.841584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:08.853830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.854825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 198/391] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.855653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.901558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.916747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56d5355-b30a-4c19-b471-d25d05d74b23 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.950776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.951404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.951686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.951963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:08.955667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ac35c-0d87-4c1b-88c9-cfd2835b5c64 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:08.959274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:08.960717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:08.972672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:08.973547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 194/392] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:08.974876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:08.989851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:08.993501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:08.994139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:08.994411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:08.994714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.002978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.004424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:09.016951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:09.017936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 199/393] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:09.018845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.078833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242eb0e-2d27-434d-aaa7-f91afe183e84 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.111669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.112299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.112582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.112862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.117461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b236ebe3-10e3-44a4-8f47-ec90bf109b2a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.123493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.124309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.124956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:09.140425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:09.141500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:09.142366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:09.142366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:09.142366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:09.142366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 195/394] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:09.142366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.155561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.156194 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.156475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.156751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.157385 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:09.157689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.159101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.170550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 200/395] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:09.171398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.208847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.247617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e529b12e-886e-4d71-810a-f80cc8a63170 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.267902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd4d0a-d15d-4fe3-b9b5-9bb9f7dbaf25 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.281341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.282138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.282500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.282787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.283479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:09.283792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.285226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.300404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.301001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.301284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.301564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.302287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:09.302629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.307915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 196/396] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:09.307915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.308122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.322196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 201/397] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:09.324354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.351136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.408570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4280884-3f51-4e47-ae5d-107c1f1c58e5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.424350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d03121-4026-4a12-9a51-d2bafb6b6464 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.431872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.443024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.443631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.443918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.444193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.444914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:09.445225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.446909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.457402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.458005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.458288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.458601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.459235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:09.459540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.460920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.472770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 202/398] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:09.473551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.488012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 197/399] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:09.488012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.560982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.577554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0a10a7-a9f2-41a2-b742-439fa0c1bbae None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.583960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ecc858-d3bf-4f37-b9e7-1da4ee20d7cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.610704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.611322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.611604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.611879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.616999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.617663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.618563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.618883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.619620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.621082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:09.632017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:09.632903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 203/400] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:09.634545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.636121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=8060598d22754154a1c7af65a96aaac2, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.637568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.650990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 198/401] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:09 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:09.651779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.690892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.762589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8d989-40a9-41a9-981c-0473b328c581 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.777886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc95d4f-4c7f-4e57-b5d0-ebdb7999c35f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:09.803260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.803909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.804188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.804459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.816538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.817537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:09.818023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.818194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:09.818489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:09.818746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:09.819389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:09.821716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:09.822197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:09.836231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:09.838151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:09.839019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:09.839019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:09.839019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:09.844107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 204/402] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:09.844107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.848329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 199/403] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:09 2022] POST /identity/v3/users => generated 247 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:09.848329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:09.962370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:09.981154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccecebb-b8b7-43a9-ac46-6a2fbb555897 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.011673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932d36c-cd92-4a1f-bfbc-48c4bc45ffb5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.021057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.021779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.022133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.022505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.023321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.023699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.025438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.047276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 200/404] 10.222.0.23 () {56 vars in 1139 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.048139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.054845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.056021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.056841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.057121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.070760 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.072622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:10.089766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:10.090730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:10.092390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:10.092390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:10.092390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:10.092390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 205/405] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:10.092390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.166926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.173158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4520248-6491-476a-b074-e846bd2c4c29 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.200405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eefcd34-5ee6-43c1-a150-95828a0650d8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.209593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.210208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.210530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.210810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.211455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.212353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.213870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.230420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 201/406] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.230420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.237222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.251554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.252592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.253099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.253575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.265973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.267787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:10.286270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:10.287252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:10.288095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:10.288095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:10.288095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:10.290725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 206/407] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:10.291759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.341179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5805-9763-42a1-a0c7-559e05a1d705 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.376472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.377090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.377372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.377645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.378425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.378604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.379983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.393241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 202/408] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.394106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.406625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.409892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450ab98-a7aa-4bf3-8fd2-1a38e09438fa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.447536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.448479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.448867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.449247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.456680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.458390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:10.472350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:10.473466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.474508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:10.479896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.481168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 207/409] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:10.482166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.520221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450ecbc-5af4-4f6e-9ee9-5ed12ac33a62 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.559748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.560620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.561046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.561463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.562423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.562955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.564653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.579434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 203/410] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.580588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.592066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2656bc06-1d8c-4d33-904a-2f0205bebf66 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.627351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.628139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.628513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.628889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.638751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.640368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:10.652852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:10.653933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.654986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:10.661073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 208/411] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:10.661073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.680610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.705329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bfa23-6de0-407e-9a56-0b4e668695f4 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.734499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.744779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.745732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.746147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.746657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.747529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.747977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.749749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.763932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 204/412] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.764929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.769877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd3f6d9-cfdc-43ab-9f40-c057bda3e2e5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.806942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.807739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.808118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.808491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.815662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.817611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:10.830346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:10.831487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:10.832509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:10.834286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 209/413] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:10.835441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.881960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f1951-a253-46a6-af6e-afbc93ade4f0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.920107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:10.923002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.923778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.924199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.924680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.925505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:10.925950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.927572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:10.940880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef1bf03-6c00-4592-a8f0-26b9d1fb8838 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:10.946696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 205/414] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:10.947784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:10.977268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:10.978083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:10.978504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:10.978915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/19022fccab844d1dac60f5eddf67f323/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:10.994977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=19022fccab844d1dac60f5eddf67f323, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:10.996580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:11.001817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:11.014975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 210/415] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:10 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/19022fccab844d1dac60f5eddf67f323/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:11.016153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:11.086093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ccfb90-00e9-4553-9d4b-c28b657e2fda None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:11.131301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:11.132314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:11.132794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:11.133289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:11.134374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:11.135108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:11.136822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:11.153274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 206/416] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:11.159898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:11.208996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081ed690-6308-4935-a340-ee5efe99065a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:11.273123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:11.285102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:11.286341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:11.286672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:11.286950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/a323cd646bae44b5a6f63714b5cdf70b/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:11.302615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=a323cd646bae44b5a6f63714b5cdf70b, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:11.303890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbc36df-fcb3-4ba7-a9a4-be028aca0608 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:11.325421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 211/417] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:11 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/a323cd646bae44b5a6f63714b5cdf70b/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:11.330887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39b34ea-76d1-4866-a0cf-84edcb0fc894 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:11.340510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:11.381957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:11.383023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:11.384232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:11.384657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/9900bcd989014ec783288c63d352d5c9/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:11.406905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=9900bcd989014ec783288c63d352d5c9, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:11.408706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639e1ac8-554f-4944-a45e-a4d08612cac9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:11.429201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 207/418] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:11 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/9900bcd989014ec783288c63d352d5c9/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:12.139137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711f445-bdd0-42db-a900-d9e7cfa62be2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:12.139678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711f445-bdd0-42db-a900-d9e7cfa62be2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:12.139678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7711f445-bdd0-42db-a900-d9e7cfa62be2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:12.141036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 212/419] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:12.150473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:12.150757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:12.151037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:12.174706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:12.175905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:12.216070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904c39-d64f-4e18-841c-6d7303c30843 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:12.216407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904c39-d64f-4e18-841c-6d7303c30843 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:12.216829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e904c39-d64f-4e18-841c-6d7303c30843 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:12.218960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 213/420] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:12.225596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:12.225880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:12.226162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:12.250004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:12.251716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:12.297461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 208/421] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:12.348202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 214/422] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:12.783494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:12.840777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:12.896965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1fc4a5-8aef-43a0-b0c0-c37ef31f18b0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:12.918470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:12.936973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:12.937532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:12.937816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:12.938095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:12.950168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:12.955747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:12.956611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:12.957391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 209/423] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:12.964168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.000523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758cde1-5a84-4178-8f72-5e9878fae034 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.026467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.076728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6577d7fc-0f43-4e3d-aabd-8a02ae3b25c0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.113372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.114181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.114610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.115016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.115917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:13.116383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.116853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.117637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.118417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.118770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.119195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.126911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 210/424] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:13.133107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.146809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.158333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:13.168059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:13.168927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:13.169695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:13.176332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 215/425] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:13.177300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.262147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa1d3e7-28d3-4232-b6e9-998276b3b4e1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.297831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.299351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.299635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.299912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.323172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.324631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.331365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837605a2-32b1-4d69-8763-c98ba22c9a58 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:13.353837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:13.354838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:13.355650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:13.355650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:13.355650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:13.357621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 211/426] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:13.364208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.397883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.398517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.398800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.399664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.400495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:13.400803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.402201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.422082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 216/427] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:13.427742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.478033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.484027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.558471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6b87e1-09fa-48a8-b951-6934ed4a350c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.587816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45e5e19-55eb-48cd-b8a1-62bbd9cd2f7d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.619425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.620045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.620326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.620600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.621251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:13.621554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.624055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.651262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 212/428] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:13.655951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.660069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.660717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.660995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.661265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.679236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.682023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:13.707354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:13.708303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:13.709122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:13.709122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:13.709122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:13.712749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 217/429] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:13.717392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.810794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec85321-1c28-4745-aaf9-24da97e34691 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.837819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.840539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:13.859191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.859898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.860182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.860469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.864772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17900c3a-1464-48c3-81b2-df351cc0cdfd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:13.868970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.870582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-de18098d-a34c-4034-942f-7a0f84121339 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:13.885149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:13.886120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:13.887477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:13.891731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 213/430] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:13.893244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f42c-dc99-4677-a6b8-66c26db66075 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.893538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f42c-dc99-4677-a6b8-66c26db66075 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.893818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a15f42c-dc99-4677-a6b8-66c26db66075 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.895635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 214/431] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:13.896586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15f42c-dc99-4677-a6b8-66c26db66075 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:13.915556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:13.917072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.917812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.918286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.919288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:13.919777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:13.921630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e848c9-669d-4adf-b932-15af2b2fea5d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:13.938795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 218/432] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:13.941186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:13.941693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:13.942148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:13.965622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:13.967833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:14.016968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a15f42c-dc99-4677-a6b8-66c26db66075 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.050497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 219/433] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:14.050497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.064220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.065347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.065712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.066063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.067336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:14.067729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.069565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319893a7-3231-4499-bcdd-b603cffd0d17 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.102235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 215/434] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:59:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:14.102235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48e3c-59aa-44a8-84f7-fe3377e647a4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.102235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48e3c-59aa-44a8-84f7-fe3377e647a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.102235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a48e3c-59aa-44a8-84f7-fe3377e647a4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.102235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 216/435] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:14.103411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80ad8-5f14-4776-adda-71fa73a72509 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.103913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80ad8-5f14-4776-adda-71fa73a72509 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.104454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b80ad8-5f14-4776-adda-71fa73a72509 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.106559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 217/436] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:14.107309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80ad8-5f14-4776-adda-71fa73a72509 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.146016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.155678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.173890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca4781a-7b4e-4c79-8594-2c2e95c1a415 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.208431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.209084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.209559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.209671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.217673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.219127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.223407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b80ad8-5f14-4776-adda-71fa73a72509 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7351094e-458f-4b7a-91de-5e52f49ada86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:14.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:14.233064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:14.234098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 220/437] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:14.238657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.238912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.239217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.256973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:14.258423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:14.263196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.263831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.264114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.264390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.275323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.277230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-80a605dc-5d19-4f2a-89aa-b1d312083dcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:14.293121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:14.294038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:14.294951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:14.294951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:14.294951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:14.294951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 218/438] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:14.299304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.299874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.300149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.318782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:14.319882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:14.330549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 221/439] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:14.331446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.396275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.403881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 219/440] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:14.405087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.427380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.449706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becc3d3b-cf84-41eb-a32d-ed2fbdbb77e0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.487118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.487794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.488082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.488360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.489086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:14.489399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.491548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.513915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 222/441] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:14.515220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.535376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6981f2-604e-4ba1-b7a8-7a12857640f9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.579158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.579690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.579982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.580256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.580975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:14.581193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.582726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.596792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 220/442] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:14.597677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.644499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326e926-8fec-4e36-90a7-afeff93e58e4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.671236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.687985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.688624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.688908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.689233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.689380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.700091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:14.704991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:14.705952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 223/443] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:14.706939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.714629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c682d3-8bfa-4e50-97cb-edfc2195bf9d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.748129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.748743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.749020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.749293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.807502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47c86c-e954-40b3-9ffe-fdca2fe50738 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:14.815486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.816913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.840480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:14.841033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:14.841313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:14.841594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:14.848292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:14.849745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:14.861289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:14.862281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:14.863323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:14.866586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 224/444] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:14.867856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.898555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.912142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:14.919471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 221/445] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:59:14.920277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:14.974004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55521-b307-4af2-98ee-4a41d2a58ef4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.006884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.007498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.007776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.008046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.017916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f13c66-d2bb-429c-9dc5-532e7e268e81 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.018332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:15.023501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.024549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 225/446] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.025544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.051591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.052337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.052714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.053078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.063501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:15.068221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.069268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:15.070103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 222/447] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.070793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.112539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.124709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42837107-9163-495d-986e-ca04cd8318c7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.126326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.158892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.159491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.159770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.160043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.160693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:15.160998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.162375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.170326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da222e6c-acc9-40c5-990d-fd311daa8090 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.170785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 226/448] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:15.171560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.204125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.204734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.205017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.205342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.206191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:15.206321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.207726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.219288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 223/449] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:15.221017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.272651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b4958a-c958-4111-acd6-3138a678ebe2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.306509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.307124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.307406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.307676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.316798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.319160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.319897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b7671d-2933-4c4d-b266-8c42af1a2f7f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.320628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:15.335472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.336481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:15.338154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:15.338154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.338154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:15.342043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 227/450] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.342762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.355365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.355977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.356261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.356539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.357194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:15.357504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.358919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.361447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.366765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 224/451] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:15.367496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.442235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3c3fa4-45ce-46db-8137-8e25c0d928f4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.468867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74abc801-91ad-4e04-bbd5-6d8f2cf6734a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.477269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.477867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.478139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.478411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.479085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:15.479387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.480766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.488518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 228/452] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:15.489283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.502984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.503617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.503901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.504178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.515166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.516639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:15.530967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.531848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:15.532703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:15.532703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.532703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:15.532703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 225/453] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.532921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.548606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.567064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.588989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08c8b01-4aca-45b8-a1bd-30f33e918a76 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.622274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.623164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.623438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.623712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/e03f5a82065a41508004af126326e8ca/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.633648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cc3203-ce27-46d7-b539-094e23f59980 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.637539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=e03f5a82065a41508004af126326e8ca, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.639102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.652850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 229/454] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:15 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/e03f5a82065a41508004af126326e8ca/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:15.653820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.669764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.671384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.671683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.671966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.672640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:15.672952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.674403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.690589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 226/455] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:15.691260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.780506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.790730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571f9f3a-3e38-44a5-83a9-69c7a6535aec None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.809489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c369cb-1341-4ca9-88cb-ae9b2d7b2570 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:15.822519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:15.832934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.833576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.833911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.834192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.845817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.847315 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.850718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:15.851838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:15.852812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:15.853229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:15.863158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.864251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:15.865079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:15.865079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.865079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:15.865079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 230/456] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.865079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.871479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:15.873004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:15.889411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:15.890340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:15.891252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:15.891252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:15.891252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:15.891252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 227/457] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:15.891475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:15.978347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79181d-c4b0-4820-b1ad-c7fde87f8b15 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.006565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23579233-a19a-4158-a0e9-de14a74f47f6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.020114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.021029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.021782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.022065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.022999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.023309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.025980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.033876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.043169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 231/458] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.043303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.043819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.043920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.044229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.044508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.051585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.053036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:16.065149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:16.066248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:16.067133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 228/459] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:16.067609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.099769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.163170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c324c9-67fa-401b-9ba2-dc871ef42e87 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.183747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59149-f036-4247-ae95-c6ebc32710e2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.200757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.201361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.201640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.201903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.202588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.202896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.204289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.219845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 232/460] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.219845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.225499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.226192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.226596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.226945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.227743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.228114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.229564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.241981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 229/461] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.244032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.293711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.332047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.336015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cb8321-6650-4336-8a4c-697b6c19873b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.357634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2816ffa2-c723-4595-b1d5-15d6fad0b731 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.374829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.375564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.375847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.376120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.384257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.385724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.394672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.395281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.395564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.395838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.396522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.396832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:16.397153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:16.397979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Oct 23 01:59:16.399827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:16.400318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.403500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 233/462] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:16.404154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.416577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 230/463] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.416577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.507984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2638f0c-be81-4442-90b8-e7ea7c07f40d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.520040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8890923-b4a2-4c5f-9d8c-f15eb15c7c0e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.527724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.546647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.547345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.547628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.547952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.556884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.557835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.558326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.558529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.558769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.559048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.566044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.567804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:16.570216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:16.571815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:16.572988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:16.576768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 234/464] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:16.577592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:16.583809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:16.584678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 231/465] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:16.590241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.600248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.681741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d5ab75-1f68-4114-85f5-8245e1187c51 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.708170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db335b-bbe4-4745-aaf1-873798f86172 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.718288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.718922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.719199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.719471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.720438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.720884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.722163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.744234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 235/466] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.744234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.749038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.749650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.749934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.750208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.750978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=8060598d22754154a1c7af65a96aaac2) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.751288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.752709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.759708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 232/467] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/role_assignments?user.id=8060598d22754154a1c7af65a96aaac2&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.760359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.796070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.867329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:16.880553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1225d7-5d9c-4278-9d2e-063cde630993 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.893987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e52ea4-fe9f-46b1-aa07-f5d15dc1d52e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:16.921186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.921822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.922154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.922458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.923211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.923526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.925093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.937513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:16.938505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:16.939012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:16.939483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:16.940425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:16.940731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:16.942184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:16.946811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 233/468] 10.222.0.23 () {56 vars in 1139 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.947564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:16.961955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 236/469] 10.222.0.23 () {56 vars in 1137 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:16.962963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.065719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25441dc-0b0d-4e11-a1fa-5949a721eba9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.080026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.087736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6e22d0-d42e-4c14-9b20-7c81f47a480f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.103991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.104920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.105416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.105878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.132690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.133995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.133995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.134295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.138740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.195764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.197691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.216914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.218713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.313882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.330771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 234/471] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:59:17.331054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 237/471] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:59:17.331627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.332495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.380534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.453867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c56ca4-0c94-432f-b35e-ada3fce1924f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.468842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30370c08-d10a-4724-bb32-376c828bc114 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.493906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.494559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.494874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.495160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.516890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.517514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.517794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.518067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.530149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.531734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:17.548326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:17.549278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:17.550100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:17.550100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.550100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:17.550100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 238/472] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:17.550309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.567885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.583883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.585988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.659757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.681488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228ff6fa-6655-4921-8946-70089b33884b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.713969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 235/473] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Oct 23 01:59:17.716091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.729387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.730776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.731169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.731545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.749458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.751319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:17.770835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:17.771958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:17.773000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:17.773000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.773000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:17.775000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 239/474] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:17.776119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.824951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.838938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d39c60-2240-448e-95ea-383b95e43b0c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.873566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.874399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.874560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.874726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.875349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:17.875647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.877019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.892905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 236/475] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:17.892905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:17.905709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8978704-17de-4beb-9d44-56a61008e735 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:17.940110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:17.940977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.941259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.941534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.951685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:17.953932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:17.955526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3122b88e-5454-4a9a-bd63-66d8bf6fa239 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:17.970874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:17.971781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:17.972625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:17.972625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:17.972625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:17.974600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 240/476] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:17.977549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af006c0-4e8d-4ab8-8b74-0e5bd038230b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:17.977844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af006c0-4e8d-4ab8-8b74-0e5bd038230b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:17.978114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af006c0-4e8d-4ab8-8b74-0e5bd038230b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:17.979444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 241/477] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:17.983764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af006c0-4e8d-4ab8-8b74-0e5bd038230b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.009455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc7c767-e89c-4e49-9a4f-6d2288313552 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.045535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.046147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.046433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.046736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.047372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.047680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.049256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.062316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 237/478] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.063184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.077312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.099326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af006c0-4e8d-4ab8-8b74-0e5bd038230b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.139994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.140399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.140679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.140950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.147722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.149962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7ea67d8a-b0ad-41e5-93cb-184885fe58cd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:18.162545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:18.163394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:18.165189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 242/479] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:18.166622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.166905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.167178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.179411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a509f9-c37f-42b4-9f61-91a3ee004cbb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.189877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:18.191425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:18.216847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.217564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.217831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.218104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.218760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.219063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.220818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.237858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 238/480] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.237858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.246010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.267691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 243/481] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:18.268795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.321924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.361155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bccd47-0ea1-4bdd-86cd-ce72cfe9a5b2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.398067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.398762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.400352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.400633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.404971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d16e54e-7050-4db8-a0b8-b6cd7ead8368 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.411423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.412888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:18.425068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:18.425928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.426762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:18.430801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 239/482] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:18.431659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.449499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.450609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.451163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.451759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.452687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.453216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.455046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.475225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 244/483] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.476379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.526105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.551142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d253c2-7fc0-4992-8db0-abc4bf2252dd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.580060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.589631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.589800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e02f95-28a9-49aa-bc8b-020d505ac283 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.590271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.590588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.590866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.598527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.600099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:18.613384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:18.614243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:18.615118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:18.619660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 240/484] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:18.620737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.625671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.626272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.626586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.626856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.627536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.627839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.629292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.640831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 245/485] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.641620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.739855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c9e5b1-9fc6-48a1-b584-e3c609ee6c0a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.761241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa86040-4919-4181-85cd-fcf43d9e5ac6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.762824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.775656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.776313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.776538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.776809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.777525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=9900bcd989014ec783288c63d352d5c9) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.777832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.779282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.785673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 241/486] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/role_assignments?user.id=9900bcd989014ec783288c63d352d5c9&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.786987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.799831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.800608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.800954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.801293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.813977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:18.819917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:18.821529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 246/487] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:18.827692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.875244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:18.936395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d12068-2202-487a-b7eb-4360699a0b24 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.954004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa032c9-1e34-4a50-b37d-5532245724f3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:18.975400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.975986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.976365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.976737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:18.978060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:18.978481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:18.980511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:18.995160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 242/488] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:18.996275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:18.997486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:18.998264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:18.998723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:18.999181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.000219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=19022fccab844d1dac60f5eddf67f323) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:19.000786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.002463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.009504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 247/489] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/role_assignments?user.id=19022fccab844d1dac60f5eddf67f323&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:19.010632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.042178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:19.125120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be19e-b672-47ca-8acc-6f5ae07207b3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.138474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97423fd4-f6de-438d-8da2-cd61733eadb3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.151563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:19.170264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.170927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.171296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.171483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.172154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:19.172807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.173936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ece6c08-8467-4d3f-9dc0-14a6cfd0fcdb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.181390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.182265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.182730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.183258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.183701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 243/490] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:19.188866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=a323cd646bae44b5a6f63714b5cdf70b) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:19.189044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326bbe5-f501-4c3a-816b-5c1e569ea29d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.189044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326bbe5-f501-4c3a-816b-5c1e569ea29d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.189044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3326bbe5-f501-4c3a-816b-5c1e569ea29d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.189044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 244/491] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:19.189966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.191406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326bbe5-f501-4c3a-816b-5c1e569ea29d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.191821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73611835-26e2-42c1-b361-7dd763d29d9f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.199368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 248/492] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/role_assignments?user.id=a323cd646bae44b5a6f63714b5cdf70b&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:19.201016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.201448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.201961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.229280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:19.230545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:19.313563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 249/493] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:19.332982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3326bbe5-f501-4c3a-816b-5c1e569ea29d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.374182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.375102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.375560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.375981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.388541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.390282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:19.406550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:19.407700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:19.409437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:19.409437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:19.409437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:19.411177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 245/494] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:19.416917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.440858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:19.525603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd57d07-5259-438e-a03a-5b2702b6760a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.574146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.575074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.575537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.575985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.576828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:19.577307 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.578905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.597487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 250/495] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:19.603784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.607908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.638421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:19.740767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:19.772432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac57cf-e0fb-46b2-a8be-7f3fc831abe8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.788894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e1011-4099-434a-8b9d-838ae3468fc8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:19.829717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.830591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.830871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.831139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.849264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:19.855008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:19.855637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:19.855916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:19.856191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:19.860004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:19.860897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:19.861871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 251/496] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:19.871437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:19.873126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:19.874611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:19.889836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:19.891668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:19.892801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 246/497] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:19.899130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.039367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c608fc-a71e-4d83-aecb-47be52b6c00a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.063152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.090119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436f29ae-7e27-468e-9876-4889dffa4d82 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.103328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.103940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.104488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.105147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:20.105447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.106845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.121196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 252/498] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:20.134265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.138319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.199302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.199942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.200229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.200505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.201250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:20.201552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.203013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.251481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 247/499] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:59:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:20.252779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed7112a-b581-4026-a8fc-c46e24de4e5d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.270751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.294791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.295427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.295708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.295983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.308070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.309546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:20.324775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:20.325743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:20.327582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:20.327582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:20.327582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:20.327582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 253/500] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:20.331484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.448227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.490090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d5bdc-8778-48fb-828b-5c0ce4e3aa81 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.506288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9572d4-b272-4543-a91c-da5f4267fe5b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.554317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.554953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.555779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.556059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.556700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:20.557007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.558396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.560383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.560990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.561269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.561540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.583636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.587203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 254/501] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:20.591833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.694098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.695106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.736507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71692a9-7641-4118-87a3-19c6389c0a01 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:20.766042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.819467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:20.820106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:20.820384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:20.820650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:20.833492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:20.834978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:20.841897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 248/502] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 572 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Oct 23 01:59:20.847743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:20.861017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:20.862222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:20.863363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 255/503] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:20.870275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:20.954044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:20.998577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c65d667-d0a2-4381-83da-e65ddceb9874 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.034418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.035408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.035926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.036398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.037338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:21.037853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.039533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.058159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 256/504] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:21.063806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.071563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77943ed0-f49d-4e1c-9bf2-52c337a9f5db None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.095946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.130226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.131139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.131342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.131525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.142782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.144292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:21.159934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:21.160964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:21.161837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:21.161837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:21.161837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:21.161837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 249/505] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:21.171521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.177949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3bbbd9-2ffb-4373-a44d-dc656ae4cd56 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.249018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.249987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.250464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.250956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.262716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.264533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:21.280219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:21.281170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:21.282052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:21.282052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:21.282052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:21.284149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 257/506] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:21.290332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.348866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.357704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6c4154-de4a-4438-bab5-7ed65e8c109b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.387141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.404927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.405627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.405978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.406319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.406953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd82ea-5a62-46bf-9c46-c2d4d7378fb3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.407062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:21.407424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.409120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886d36ab-1f8e-4cab-ba67-e2c58b2bfb0d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.434627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 250/507] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:21.436191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e133e9-edc5-4fbe-b79c-6b565463538b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.437844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e133e9-edc5-4fbe-b79c-6b565463538b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.438120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e133e9-edc5-4fbe-b79c-6b565463538b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.439834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 251/508] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:21.440179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e133e9-edc5-4fbe-b79c-6b565463538b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.457408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.458908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.459383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.459852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.460832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:21.461932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.464225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f68124-ad1d-420f-93a3-b6af29fd8847 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.478813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 258/509] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:21.480333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.480833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.481297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.503087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:21.504812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:21.563725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e133e9-edc5-4fbe-b79c-6b565463538b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.586969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 259/510] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:21.588444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.604700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.604700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.604700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.604700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.610978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.612836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:21.627074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:21.627981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:21.628858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 252/511] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:21.634248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.659353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.740737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.762051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c008f4ec-55c2-4fe7-bf5a-5858d6a4ea0e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.806575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.808387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.808669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.808943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.809667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:21.809976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.811484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.815365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80a7e7-5ec8-4065-a617-0fa91878b3d7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:21.846570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 253/512] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:21.853156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.887728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:21.888403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.888681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.888952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.897252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:21.899650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-59906c0b-f36c-4e7a-89c2-d4e4e9e58a1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:21.913535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:21.915207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:21.916501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:21.917834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 260/513] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:21.919254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad226a-f18b-4055-9363-c1d487220655 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.919688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad226a-f18b-4055-9363-c1d487220655 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.920088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ad226a-f18b-4055-9363-c1d487220655 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.921496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 261/514] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:21.922655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecbd48-d0af-4e92-8d3e-f4a8c911a624 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:21.923073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecbd48-d0af-4e92-8d3e-f4a8c911a624 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:21.923522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ecbd48-d0af-4e92-8d3e-f4a8c911a624 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:21.925250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 262/515] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:21.926087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecbd48-d0af-4e92-8d3e-f4a8c911a624 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:21.959332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:21.984900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b458b31-85d5-4fb9-96f2-440ce04d089f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.018838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.019371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.019653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.019925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.020643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=e03f5a82065a41508004af126326e8ca) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:22.020952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.022369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45e80ab-171d-4644-9a7d-a72863bfe687 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.028807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 254/516] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/role_assignments?user.id=e03f5a82065a41508004af126326e8ca&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:22.029932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.030218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.030523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.036182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ecbd48-d0af-4e92-8d3e-f4a8c911a624 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.050376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:22.051645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:22.075781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.076398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.076680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.076955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.077652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:22.077961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.079466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b6167b-6f30-4044-a3da-7b1bada13cbc None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.081336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.094047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 263/517] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:22.094047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.094047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.094047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.116582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:22.117920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:22.142803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 255/518] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:22.143637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.210698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 264/519] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:22.211320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.238062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.276942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5739c1f1-d037-4890-a1fc-7c23c59152a9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.323104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.323420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.323683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.323957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.324903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.326007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.338772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022afdb-f04e-42bf-83fc-8dcf848a33c6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:22.339934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:22.340880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.342946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 256/520] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/services/network => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:22.347425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.370946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.383528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.384591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.385066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.385534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.387693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=8060598d22754154a1c7af65a96aaac2, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.389334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.407601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 265/521] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:22 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:22.421759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.484429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c8f46-8da6-4202-aa04-2f179c54b72e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.524374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.524997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.525276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.525595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.526221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] RBAC: Adding query filter params (name=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:22.526588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.528096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.529324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.534183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 257/522] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:22.539720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.612016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c581e02-8c91-4a15-887f-f13826528bdf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.659721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aa9b57-3015-42f2-b7bc-fa276937003b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.661732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.662364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.662719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.662995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.676745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.688306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.700753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.701371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.701654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.701931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.702590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] RBAC: Adding query filter params (type=network) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:22.702946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.704330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:22.709611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:22.710611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:22.711631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:22.711631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:22.711631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:22.711811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 258/523] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/services?type=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:22.711811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.713968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 266/524] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:22.720890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.790927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.849054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:22.909265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dea2df3-62ee-4b80-9080-c8a96b388f8c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.916840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c401ad-2faa-4654-9e42-b65cb48cdda7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:22.957605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.958380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.958753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.959125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.959844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:22.960155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.961097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:22.962085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.962263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:22.962571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:22.962853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:22.963427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:22.964798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:22.974407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:22.975430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:22.976286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 259/525] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:22.978827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 267/526] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:22.982503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:22.984486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:23.106927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9619f51-5a56-4095-8cbc-f539d1b45147 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:23.118164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:23.132741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d76d0-061d-4036-a220-6ebecd4c4bfe None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:23.157252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:23.173388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:23.174327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:23.175211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:23.176003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:23.176478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:23.176964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:23.177444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:23.177906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:23.178763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:23.179266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:23.181403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:23.183216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:23.184651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7dda9b-1981-4e18-ba63-83633a3c258c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:23.187099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 260/527] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:23.194474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:23.216811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 268/528] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 01:59:22 2022] POST /identity/v3/users => generated 254 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:59:23.328589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ee277-7e46-47d9-9eb5-0b176651d99c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:23.378189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:23.379248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:23.379735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:23.380212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:23.381088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:23.381595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:23.383932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad808e2-7f85-4657-bb88-a4d86a31cca4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:23.390079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 261/529] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:23 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:24.717929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f98bc-b9d6-4022-b909-ca0332345fff None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:24.718458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f98bc-b9d6-4022-b909-ca0332345fff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:24.718541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f98bc-b9d6-4022-b909-ca0332345fff None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:24.720191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 269/530] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:24.725284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0446-5056-44fe-a6c1-50503ebbd225 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:24.726121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0446-5056-44fe-a6c1-50503ebbd225 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:24.726639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873a0446-5056-44fe-a6c1-50503ebbd225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:24.753081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-873a0446-5056-44fe-a6c1-50503ebbd225 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:24.754690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-873a0446-5056-44fe-a6c1-50503ebbd225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:24.824197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:24.845955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 262/531] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:25.270417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40310c64-50d0-46d3-af8b-3bdde983f8b5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.270729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40310c64-50d0-46d3-af8b-3bdde983f8b5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.271007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40310c64-50d0-46d3-af8b-3bdde983f8b5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.272645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 270/532] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:25.279034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.279324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.279599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.296565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40310c64-50d0-46d3-af8b-3bdde983f8b5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:25.303854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:25.305558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:25.442022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 263/533] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:25.460800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40310c64-50d0-46d3-af8b-3bdde983f8b5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:25.496460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:25.497102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.497384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.497659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.509145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-92ed05ac-4477-4c51-a1b3-5385730ca8f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:25.514217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:25.515159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:25.515975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 271/534] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:25.522959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:25.543693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c473b-42df-4e06-aab6-494264477f30 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.543983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c473b-42df-4e06-aab6-494264477f30 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.544250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c473b-42df-4e06-aab6-494264477f30 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.545875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 272/535] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:25.550919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.551203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.551479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.583680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:25.584872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:25.599822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:25.690904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bd292-f4de-40b9-a889-b1852fb33f27 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:25.722856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:25.735997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:25.736630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.736893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.737160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.737837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:25.738145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:25.739593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99beaf5-55d5-4253-9b67-49e5bbaf7ab6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:25.744231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 273/536] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:25.758098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 264/537] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:25.759798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:25.870325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df62c0-3611-482b-b243-9b2780bdefef None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.870683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df62c0-3611-482b-b243-9b2780bdefef None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.870925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75df62c0-3611-482b-b243-9b2780bdefef None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.872512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 265/538] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:25.878375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.878695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.878973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.884889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c13d66-4ea4-4567-bcf2-a7d962b5662a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:25.915294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:25.916495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:25.924116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:25.924933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:25.925805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:25.926233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:25.939721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:25.941870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:25.944202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:25.964954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:25.970670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:25.971557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:25.971557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:25.971557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:25.971557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 274/539] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:25.971557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.002528 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 266/540] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Oct 23 01:59:26.003707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.118788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c3f8e4-d839-4e88-b755-e3f6e2ea8407 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.144551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86026dca-db9e-4020-94e7-6489882ea052 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.155060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.161627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.162535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.162981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.163444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.175715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:26.182191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:26.183274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:26.184289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:26.191468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.191468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.191468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.191468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.191872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:26.191872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.191872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.192383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 275/541] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:26.193537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.209387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 267/542] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:26.210381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.254818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.303409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401284b1-226e-4606-b8c1-f01a515acb85 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.317992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba647a-f1fd-4c59-bf77-ddd8abbfc5e2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.337053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.337660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.337953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.338210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.338882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.340339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.352370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.352969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.353252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.353527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.354183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:26.354512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.355874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a798f841-8658-4b83-b3a3-4ac360fb1769 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.358679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 276/543] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:26 2022] POST /identity/v3/services => generated 240 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:26.358679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.366952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 268/544] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:26.369067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b78a-6d83-44cc-bbb1-9883dd800dc8 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.369526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b78a-6d83-44cc-bbb1-9883dd800dc8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.369978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d9b78a-6d83-44cc-bbb1-9883dd800dc8 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.371887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 269/545] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:26.372853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b78a-6d83-44cc-bbb1-9883dd800dc8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.410946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.485516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7de4f1-cacb-485e-a152-e8f5dfb166e9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.496929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.500188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d9b78a-6d83-44cc-bbb1-9883dd800dc8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.528980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.529438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.529721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.529995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] PATH_INFO: `/v3/users/swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.537867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.539899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.542217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.542871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.543152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.543423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:26.552584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:26.553504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 277/546] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:26.554430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.556506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-a72ce510-bb4b-4d9d-983e-04479f1e2629 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:26.562114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:26.563093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:26.563917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 270/547] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:26.565131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.567308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.567308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.588476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:26.589689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:26.657854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 271/548] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:26 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Oct 23 01:59:26.658726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.672929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf0bc6-dba1-4a9f-8c0a-bf66da5841ca None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.690918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.726043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.727992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.728967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.729637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.744645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.747426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:26.769882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:26.771143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:26.772221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:26.772221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:26.772221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:26.775070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 278/549] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:26.776085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.801983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:26.804424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22beceab-b046-4021-84c6-e4f963ae5b98 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.853280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.854418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.854878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.855298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.857429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:26.857915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.859517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.887495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 272/550] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:26.887495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.897955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e055799-1e0f-477d-a250-2e429267ea02 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:26.946090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:26.947108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:26.947403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:26.947910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:26.948561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:26.948865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:26.950272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:26.959380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 279/551] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:26.960274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:26.978989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.004864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9ee6b-8db3-47d4-9ac6-774dddfcbe2b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.041856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.042483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.042773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.043047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.043693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:27.043979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.045352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.049460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.062816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 273/552] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:27.063674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.067458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b77a97-f887-4ff5-93b1-65c62921a6d4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.103616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.104255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.104542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.104820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.115567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.117025 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:27.131320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:27.132467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:27.133898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:27.133898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:27.133898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:27.133898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 280/553] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:27.133898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.174073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adef2d-42bd-4cde-83d1-da3a53fba048 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.212846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.213376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.213660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.213937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.214616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.215976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.222027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.235761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 274/554] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:27 2022] POST /identity/v3/services => generated 251 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:27.239885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1628313-df3d-415a-854e-9e09e9db2309 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.243653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.283244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.288638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.289310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.290122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.290395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.303280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.304830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:27.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:27.323079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:27.323967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:27.323967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:27.323967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:27.327933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 281/555] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:27.328957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.413394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c76e4a-e686-4b95-a1d3-32f9433cf3fa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.454721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd7b2b9-f2ad-4fa1-8538-1fc58da262f9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.455435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.456173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.456458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.456732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.466576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.467186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:27.479494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:27.480625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 01:59:27.481768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:27.483957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 275/556] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:27.484907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.499418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.500035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.500317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.500590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.501226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:27.501530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.503064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.516269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 282/557] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:27.517158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.551817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.568375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.607550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b76f4e-21fc-444c-9d8c-187f570b6cef None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.630005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8abf4-08a2-486e-8330-e174473a207d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.647371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.648550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.649844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.650318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.651568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:27.652062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.653920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.671942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 276/558] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:27.672865 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.679374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.680614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.680935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.681275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.682177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:27.683044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.684602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.707036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 283/559] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:27.707966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.810359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4be6ad-6ce7-41ce-a8c7-18ac18182eaf None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.821735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805658c8-b24c-4d13-9ce2-f7e4aad6b486 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:27.830061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.840081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:27.849437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.850345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.850667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.850946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.859110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.860571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.863807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:27.864503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:27.864785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:27.865066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:27.872629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:27.874614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 277/560] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:27.876662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.882523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:27.882523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:27.897357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:27.900237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:27.902391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:27.903638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 284/561] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:27.903880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:27.993465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da6f66b-99df-4b41-9a45-ec4af4404a5f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.038686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.039037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657a2ef9-47c6-4b6c-bb99-4233e843ba58 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.040449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.041028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.041529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.075712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.076021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.076254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.076525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.077296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:28.077504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.079144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.096166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.097072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 285/562] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:28.098107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.124627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.126135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.131200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.242059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961cf45b-6442-4349-97e9-731eef274c4c None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.254416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 278/563] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 01:59:28.254416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.292058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.292689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.292968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.293240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.294012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:28.294254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.295706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.317878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 286/564] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:28.317878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.355379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.385021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95ef3b-1c58-4998-8b0c-cd26a8e42b0f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.402221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.422784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba593711-7e97-4b76-a63c-34cffb0a3660 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.429576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.431535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.432050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.432955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/e03f5a82065a41508004af126326e8ca/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.435768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=e03f5a82065a41508004af126326e8ca, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.437656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.454876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 279/565] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:28 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/e03f5a82065a41508004af126326e8ca/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:28.456578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.468021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.468795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.469078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.469347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.490782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.493043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:28.520588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:28.521592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:28.527032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:28.527032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:28.527032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:28.527032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 287/566] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:28.532793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.596204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7564db72-8262-43b7-b33c-7c5826be30fe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.638604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07434025-b413-4945-a340-d7643068143f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.639646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07434025-b413-4945-a340-d7643068143f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.640457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07434025-b413-4945-a340-d7643068143f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.640932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07434025-b413-4945-a340-d7643068143f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.644508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.657392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07434025-b413-4945-a340-d7643068143f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.662374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07434025-b413-4945-a340-d7643068143f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-07434025-b413-4945-a340-d7643068143f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:28.682348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:28.683641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:28.684722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:28.684722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:28.684722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:28.686033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 280/567] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:28.691708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3109ba-5170-4f22-9afb-c75848859714 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.692742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3109ba-5170-4f22-9afb-c75848859714 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.693025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3109ba-5170-4f22-9afb-c75848859714 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.694589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 281/568] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:28.700901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3109ba-5170-4f22-9afb-c75848859714 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.727370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ac5d55-a466-4b47-b60a-a51e0eb5e1ef None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.767149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:28.775789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.776399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.776679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.776949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.777701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:28.778120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.779547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc6a297-668d-4312-9e87-d976ad34c3d7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.793869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 288/569] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:28.795325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.795611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.795886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.814329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:28.815685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:28.833064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3109ba-5170-4f22-9afb-c75848859714 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:28.874148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:28.874574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:28.874855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:28.875127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:28.875807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:28.876107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:28.877587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:28.891987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 282/570] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:28.893202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.903502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 289/571] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:28 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:28.911532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:28.945804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.007200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfbea5e-89fe-4b11-8f59-efe065cc5e01 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.035673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580c0844-f4cd-4124-9468-dab13a97949c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.054328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.056956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.057540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.061415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.074244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.076021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.084112 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:29.091317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:29.092448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:29.093548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:29.095924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 283/572] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:29.101311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.114941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.115868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.116307 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.118860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.131351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.136216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:29.158668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:29.159588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:29.160484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 290/573] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:29.168299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.281986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69460717-09e8-4ddb-a1d9-2eef6ecf574b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.291858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.323582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.325078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.325513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.326196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.327070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:29.327815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.330047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.331136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e8a7c2-8f0a-4ab3-869f-0395838a05c6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.346640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 284/574] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:29.351603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.381638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.383577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.384129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.384580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.386052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:29.386727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.388454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.412627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 291/575] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:29.412627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.469950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7933fc-4a4b-4f1e-96aa-8633c481fb0b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.475082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.513104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.513783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.514745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.514745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.515167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.516615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:29.526532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:29.527731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:29.528671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:29.532503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 285/576] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:29.533922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.554341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10046b8d-e2ba-4ca5-8903-5b9759c9c904 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.568497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.605880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.606680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.606798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.607054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.607782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=8060598d22754154a1c7af65a96aaac2) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:29.608091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.609512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.618565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 292/577] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/role_assignments?user.id=8060598d22754154a1c7af65a96aaac2&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:29.618565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.667971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd337c1-4e7e-4be5-9f33-18233d7cece1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.721031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.721981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.722724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.723151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/768efb731b82485ba4e0a51901710a16/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.740324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9755a979-fed9-4180-a46e-16bb06bac97f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.747068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=768efb731b82485ba4e0a51901710a16, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.748774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e6a47b-2516-4bcb-b491-f700471939c4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.750235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:29.775119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 286/578] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:29 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/768efb731b82485ba4e0a51901710a16/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:29.775119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661618-1e84-4209-8be8-1788c87ef9ad None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.775119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661618-1e84-4209-8be8-1788c87ef9ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.775119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee661618-1e84-4209-8be8-1788c87ef9ad None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.776815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 287/579] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:29.784616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.784946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.785166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.805866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.806558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.806894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.807166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.807808 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] RBAC: Adding query filter params (name=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:29.808106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.811346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:29.821546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 293/580] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:29.823803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:29.825140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:29.827596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:29.916897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 288/581] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:29.949906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901de8d5-659c-4fcb-b719-d2ae93b73c46 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:29.988989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:29.989593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:29.989873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:29.990143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:29.990796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] RBAC: Adding query filter params (type=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:29.991097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:29.993036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.001267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 294/582] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:29 2022] GET /identity/v3/services?type=network => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:30.005534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:30.033504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:30.140530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88fa9a2-a2be-4ac7-be80-6d9562f84b79 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:30.188928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:30.189901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:30.190379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:30.190930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:30.191822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] RBAC: Adding query filter params (interface=public, service_id=57c62d7755a14ff78229b9bb4c069203, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:30.192329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:30.193885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.207790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 289/583] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/endpoints?service_id=57c62d7755a14ff78229b9bb4c069203&interface=public®ion_id=RegionOne => generated 194 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:30.214062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:30.273135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:30.282472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:30.409951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da6f7af-2c06-475b-8c28-5b1ec9ab5c64 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:30.439477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:30.483956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4648b656-ba25-476e-ae1d-12750ef67e99 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:30.486115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:30.486883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:30.487652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:30.487652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:30.490459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:30.490459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.505953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 295/584] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/services => generated 772 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:30.525014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:30.525677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:30.525956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:30.526342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:30.526975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:30.528398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:30.538794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:30.540272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:30.541352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:30.544618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 290/585] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:30.549842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:30.663453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:30.667860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84758a27-28ce-4f31-a31d-f9eaea3b0d9a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:30.707126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:30.707930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:30.708214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:30.708488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:30.709119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:30.709420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:30.710873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.721491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 296/586] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:30.722923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:30.841887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c6f619-3392-432d-9030-0d843b0c968d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:30.881409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:30.882025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:30.882301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:30.882603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:30.883932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:30.884455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:30.885646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:30.897019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 291/587] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:30.898494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:31.014539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642f867f-e419-4278-a4bd-3eda74d135fd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:31.053966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:31.054604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:31.054962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:31.055206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:31.055905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] RBAC: Adding query filter params (interface=public, service_id=0b14ee7a2b0b4ea99b79726e930744da, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:31.056208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:31.057570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c3b66f-8364-439b-94ac-67f8225410bb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:31.069890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 297/588] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:30 2022] GET /identity/v3/endpoints?service_id=0b14ee7a2b0b4ea99b79726e930744da&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:31.072335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:31.077124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:31.205541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b0eb4-2e01-486a-a841-f10c65f90bfb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:31.214644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270039c-f3ff-41c4-a814-f0e8645f3b3c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:31.214947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270039c-f3ff-41c4-a814-f0e8645f3b3c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:31.215203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7270039c-f3ff-41c4-a814-f0e8645f3b3c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:31.216810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 298/589] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:31.221956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:31.222236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:31.222532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:31.249665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:31.250881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:31.283835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:31.284456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:31.284735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:31.285008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:31.285610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:31.290985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:31.296427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:31.304455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 292/590] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:31 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:31.360026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 299/591] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:31.698129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:31.777985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:31.823988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75926063-5769-4726-a0f9-3f919568a03c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:31.870821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:31.871528 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:31.871817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:31.872090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:31.893406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:31.899060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:31.900060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:31.900958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 293/592] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:31.906339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:32.022093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca005e3-180d-41df-a8af-187fb8323208 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:32.057955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:32.058607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.058890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.059168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.059890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:32.060201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:32.061588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:32.070423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 300/593] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:32.076327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:32.190021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65819e6f-73d8-4475-aa5b-63fa3aa07916 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:32.217268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:32.225764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:32.226392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.226833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.227605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.238656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:32.240124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:32.256443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:32.257446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:32.258492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:32.258492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:32.258492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:32.258492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 294/594] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:32.264309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:32.291066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:32.395145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9ea832-efa2-4f4f-9ee8-79ecc0dc8478 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:32.445053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:32.446285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.446593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.446869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.447519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:32.447825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:32.449227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f95da5e-7e6d-4602-b718-d571d94bca16 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:32.469790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 301/595] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:32.475872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:32.584217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af163-0334-4609-a9d1-d84e38b95630 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.584840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af163-0334-4609-a9d1-d84e38b95630 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.585332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730af163-0334-4609-a9d1-d84e38b95630 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.587306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 302/596] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:32.593100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117e9f-fce7-42e0-bfe7-31665fc42d3c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.593394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117e9f-fce7-42e0-bfe7-31665fc42d3c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.593672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82117e9f-fce7-42e0-bfe7-31665fc42d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.621569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9234f8-388e-4234-b667-9403c14c054d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:32.627982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-82117e9f-fce7-42e0-bfe7-31665fc42d3c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:32.629163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-82117e9f-fce7-42e0-bfe7-31665fc42d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:32.672707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:32.673351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.673639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.673918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.685615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:32.685615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d5f70036-14c9-4eb0-b071-cfe5aefbd670 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:32.701238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:32.702347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Oct 23 01:59:32.703670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:32.706553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 295/597] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 01:59:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:32.709048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c16b-6b34-41a5-a866-e6f38666d317 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.709327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c16b-6b34-41a5-a866-e6f38666d317 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.709594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e7c16b-6b34-41a5-a866-e6f38666d317 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.711705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 296/598] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:32.712498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c16b-6b34-41a5-a866-e6f38666d317 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:32.732098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 303/599] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:32.733755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f793-d394-4e34-87db-252d349c3898 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.734248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f793-d394-4e34-87db-252d349c3898 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.734789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748f793-d394-4e34-87db-252d349c3898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.755759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1748f793-d394-4e34-87db-252d349c3898 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:32.757313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1748f793-d394-4e34-87db-252d349c3898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:32.759734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:32.817388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:32.877990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 304/600] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:32.921958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e7c16b-6b34-41a5-a866-e6f38666d317 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:32.957914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:32.958911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:32.959423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:32.959971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:32.960962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:32.962114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:32.963901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:32.986471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 297/601] 10.222.0.23 () {56 vars in 1135 bytes} [Sun Oct 23 01:59:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:32.992056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f793-d394-4e34-87db-252d349c3898 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.064693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.170258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1748f793-d394-4e34-87db-252d349c3898 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.214804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.218510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14540a7f-6b12-48b5-b236-0f7ff960c34d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.233059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.233671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.233949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.234220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.243948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.278097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.279384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.279947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.281239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.282235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.285084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:33.298293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:33.299386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:33.300276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:33.304044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 298/602] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:33.309977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.313950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.315447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.426232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 305/603] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:32 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 01:59:33.427034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.427405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8352b46-7bb0-4553-953c-1bd5cd6d4934 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.466894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.467560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.467849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.468120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.470713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.479219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-746bbe15-c46c-48ed-bdaf-05ad493f2874 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:33.484744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:33.485872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 01:59:33.486851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:33.490102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 299/604] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:33.491591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fab449-8656-4628-8ca0-b96ddf6cd7af None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.491865 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fab449-8656-4628-8ca0-b96ddf6cd7af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.492139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fab449-8656-4628-8ca0-b96ddf6cd7af None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.498294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 300/605] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:33.498294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fab449-8656-4628-8ca0-b96ddf6cd7af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.526518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.539797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bbefa-091c-4c4b-ad82-064a92803353 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.573871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.574496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.574778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.575048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.575662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:33.575963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.577324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.581907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 306/606] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:33.582774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.625154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fab449-8656-4628-8ca0-b96ddf6cd7af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.667754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.669081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.669542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.670004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.685527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b59ee-3ee5-4f56-96f3-d001a161fdb5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.687345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.687817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.689301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b71daaab-efce-440d-b3b1-46687fe16959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:33.704463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:33.705426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 301/607] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.712076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.726331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:33.727663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:33.730559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.731170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.731450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.731723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.732378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:33.732680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.734056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.742852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 307/608] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:33.743919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.802758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 302/609] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:33.808026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.818795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.850328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20f0b3c-a1a6-4e37-838a-a86dddc66a27 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.887111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.887860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.888214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.888562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.889248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:33.889620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.891089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:33.902935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 308/610] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:33.904000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:33.943383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05acf1d0-15d3-4c86-9cda-db413493f5c5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:33.971864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:33.988166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:33.989137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:33.989592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:33.990066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:33.990922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:33.991400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:33.993173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.012972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 303/611] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.014700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.092314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.139455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325cf5ee-42a0-4d8e-a769-6a1989964fe9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.141779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2114dea9-264a-4589-9a68-cd0cdb7eca11 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.183967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.184607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.184891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.185164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.185611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.186400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.186707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.186979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.193005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.194664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.199822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.201349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:34.208272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:34.209331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 304/612] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:34.214572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:34.227713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:34.228690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:34.229558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:34.229558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:34.229558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:34.229734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 309/613] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:34.230829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.315343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.319619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd507faa-e961-44ed-bffc-2794ec1982b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.335169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0476b-872f-4e31-be50-c1ebccccc843 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.347960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.362802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.363436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.363969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.364267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] PATH_INFO: `/v3/services/network` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.365567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.367119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.372404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.373011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.373289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.373560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.374239 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:34.374575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.375985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0d38639c-8e43-48a8-b449-622418eaa0ac None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:34.377778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:34.378934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:34.379863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 305/614] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:34.380967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666cec-0ee4-4bf1-bd41-4c946f39e3b1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.381256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666cec-0ee4-4bf1-bd41-4c946f39e3b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.381540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3666cec-0ee4-4bf1-bd41-4c946f39e3b1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.382799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 306/615] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:34.383636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666cec-0ee4-4bf1-bd41-4c946f39e3b1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.398053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 310/616] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.398053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.499966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb2ac4a-2ba0-49be-a124-f57c01a3c7ec None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.500890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3666cec-0ee4-4bf1-bd41-4c946f39e3b1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.537067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.537681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.537938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.538202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.538863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:34.539165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.540954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.541089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.541614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.541899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.542171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.542999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] RBAC: Adding query filter params (name=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:34.543769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.545147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72240472-902a-4fbb-bfeb-8b46cdd9b223 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.560395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 311/617] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.560395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.560622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 307/618] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.560801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.560801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.560932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.565679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.579938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:34.581522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:34.610248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.660503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 312/619] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:34.662268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.667609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71f6181-2952-45b8-9098-9157e122eb78 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.716416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.717214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.717588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.717963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.718806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=e03f5a82065a41508004af126326e8ca) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:34.719217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.721412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.735102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 308/620] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/role_assignments?user.id=e03f5a82065a41508004af126326e8ca&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.737022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.857483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce472b-2339-4b1f-bbd5-36a775e14ff2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.887993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407de0bd-3ba1-4008-927c-b7569c33d1c5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:34.904093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.912420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.913060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.913338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.913610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.914849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] RBAC: Adding query filter params (type=network) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:34.915156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.916744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.918386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:34.923659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 313/621] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/services?type=network => generated 350 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:34.930038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:34.936656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:34.937689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:34.938179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:34.938734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:34.948056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:34.949911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:34.963270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:34.964298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:34.965227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:34.967747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 309/622] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:34.973793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.049721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7bd83a-406a-4f39-995c-668d11b2e4ac None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.093806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.094408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.094706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.094977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.095926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.097276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.118504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60afe60-eab6-4475-bcbb-497a22f05fb3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.125355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 314/623] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:34 2022] POST /identity/v3/endpoints => generated 337 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:35.126351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.186714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.187344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.187621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.187892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.188649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:35.188947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.191051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.206675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.215363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 310/624] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Oct 23 01:59:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:35.220950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.263880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014a311-4dc9-4fd8-a2ce-a1068098dd25 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.302822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.303859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.304363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.304565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.305506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.306477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.307849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:35.318660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:35.319696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:35.320997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 315/625] 10.222.0.23 () {56 vars in 1129 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:35.326420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.417647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbdbb7-cd09-4296-8c19-b81f9f0160fc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.448719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da53ef00-8a05-441a-a46c-86563a53e36b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.463975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.464851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.465305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.465782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.478559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.497338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.499019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.499313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.499594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.500233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:35.500544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.502203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.508521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 316/626] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:35.515024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.559638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.561121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.594919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.632211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f42c201-cb8d-42c4-8c95-9bf22aa00a8e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.671907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.672946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.673434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.674326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.676488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:35.677001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.678624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.679621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 311/627] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 459 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 01:59:35.684894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.696746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 317/628] 10.222.0.23 () {56 vars in 1124 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:35.702964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:35.754996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.810596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62eb155-8da6-4a28-aea9-dd2b3bc67190 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.813238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddf9cf1-91a2-4028-b4bf-6b00792f2804 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:35.841827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:35.852711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.853441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.853832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.854195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.854677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:35.855097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.855642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:35.856070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:35.856486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:35.856822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.869529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:35.871679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:35.886559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 318/629] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:35 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:35.889665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:35.890872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:35.892014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:35.892014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:35.892014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:35.893934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 312/630] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:35.900111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:36.019095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:36.025006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9e675-27b3-4959-8966-9ca64a1d125a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:36.067170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:36.068218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.068690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.069152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.070258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:36.071300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:36.072922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:36.085600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 319/631] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:36.091636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:36.199573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fcb4ab-890a-42f4-b628-fe244bc5a64a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:36.238876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:36.240273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.240734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.241181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.249612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:36.251349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-ba6a23ee-63e1-41b6-a6de-2cd3300424b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:36.269410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:36.270309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:36.271246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 313/632] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:36.273618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:36.326215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa887-248b-4dfc-a829-e67c4828331a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.326593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa887-248b-4dfc-a829-e67c4828331a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.326859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594aa887-248b-4dfc-a829-e67c4828331a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.328324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 314/633] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:36.333563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.333847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.334122 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.356608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:36.357445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:36.388169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affcabb1-0173-4ce6-961a-89253dd78145 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:36.390483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:36.425033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:36.425758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.426109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.426471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.427258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:36.427690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:36.429446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:36.442622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 320/635] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:36.442818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 315/635] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:36 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:36.448094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:36.478823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:36.557485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47c2df5-27b0-4e20-9c93-32a0d043e2f5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:36.592155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:36.592797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.593081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.593359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.594078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=768efb731b82485ba4e0a51901710a16) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:36.594466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:36.596169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e8f222-e3f1-4a08-b8be-5eb6362a4580 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:36.602353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 321/636] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:36 2022] GET /identity/v3/role_assignments?user.id=768efb731b82485ba4e0a51901710a16&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:36.728695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:36.838149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9682a55d-be9c-4075-adef-1797c3132e94 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:36.874742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:36.875351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:36.875633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:36.875903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:36.876565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:36.877923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0bc95af-9264-4952-9311-76d17f02ab0f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:36.894370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 316/637] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:36 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:37.279749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05d36-0ef8-409e-97bc-db85574b13d6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:37.280262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05d36-0ef8-409e-97bc-db85574b13d6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:37.280340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb05d36-0ef8-409e-97bc-db85574b13d6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:37.281913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 322/638] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:37.287227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:37.287662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:37.287858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:37.306857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:37.308068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:37.332083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:37.389317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 317/639] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:37 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:37.658393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb05d36-0ef8-409e-97bc-db85574b13d6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:37.747204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:37.768464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb05d36-0ef8-409e-97bc-db85574b13d6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:37.803835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:37.804530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:37.804945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:37.805364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:37.806107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:37.807547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:37.816767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:37.817753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:37.818648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 323/640] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:37.824690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:37.936235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f30dc5b-5fd5-40b7-a88c-eb07753d7313 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:37.971960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:37.972562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:37.972886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:37.973159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:37.973782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:37.974078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:37.975466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46af9b78-c514-4323-a4ce-6654e1a9b582 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:37.981099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 318/641] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:37 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:37.986368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:38.091593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21bb17-37ea-4d7c-9f8b-6f03756aa177 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:38.131119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:38.132092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.132578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.133133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.133958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:38.134482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:38.136068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced73368-9653-45f3-a154-921ed3f57566 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:38.141146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 324/642] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:37 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:38.292501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812b09e-03f2-4777-a9ee-c346e9875a2b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.293195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812b09e-03f2-4777-a9ee-c346e9875a2b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.293195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0812b09e-03f2-4777-a9ee-c346e9875a2b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.295145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 319/643] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:38.300639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.301524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.302013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.324292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:38.325907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:38.329989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:38.399822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 325/644] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:38.680018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812b09e-03f2-4777-a9ee-c346e9875a2b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:38.703111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:38.801470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0812b09e-03f2-4777-a9ee-c346e9875a2b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:38.838292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:38.838955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.839236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.839507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.840087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:38.841449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-73ea3288-aa54-4301-a9ad-dea8bde1e7d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:38.850798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:38.851913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:38.852733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 320/645] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:38.860316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:38.864907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf12b-2bbd-4c0f-9c40-e6638befa2a2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.865193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf12b-2bbd-4c0f-9c40-e6638befa2a2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.865470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524cf12b-2bbd-4c0f-9c40-e6638befa2a2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.867067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 321/646] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:38.872977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:38.873262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:38.873537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:38.892588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:38.893717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:38.972086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d9e6e-250d-4b3f-a014-98d1ede1431e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:38.976059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 322/647] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:38 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:39.011326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.012362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.012848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.013388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.014222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:39.014764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.016407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.021816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 326/648] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:38 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:39.028222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.043539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.135885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e276c9-a925-44c2-bf36-9a7710d6f745 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.177276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.177887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.178169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.178460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.179095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:39.179402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.180906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f3c70-1c6a-44e3-9019-67371c743893 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.185682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 323/649] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:39.252692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.257619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.259615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3fab3-ca74-4354-8ce7-f1a3fa3badd1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.259921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3fab3-ca74-4354-8ce7-f1a3fa3badd1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.260202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a3fab3-ca74-4354-8ce7-f1a3fa3badd1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.262223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 324/650] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:39.268003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.268305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.268575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.289801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:39.291711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:39.377362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 325/651] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:39 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:39.377959 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad00e01-994c-4037-9d89-f46a389331e2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.417813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.418855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.419357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.419830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.431210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:39.436348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:39.437276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 01:59:39.438101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:39.440670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 327/652] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:39.446857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.459166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.555192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2466e5b-dee9-487c-8a91-2eedb4217e7f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.590613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.591219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.591496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.591768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.592417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:39.592718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.594076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.602982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 326/653] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:39.608433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.647107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.674285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.729276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c4e73b-a920-4779-8059-94ac89bd97b5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.738314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.776446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.777252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.777544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.777821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.786881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48cd5f4-17ee-4ebc-9cec-f1c7bd361891 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.791556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.793037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:39.813468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:39.814474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:39.815489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:39.815489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:39.815489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:39.815489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 328/654] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:39.823431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.833557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.834264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.834486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.834782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.835364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.836807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:39.846182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:39.847153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:39.848090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 327/655] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:39.855282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:39.946125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d7dad-cc8f-44c8-8ef1-cbaed78c9d63 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.959964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03df40ed-9faf-4d43-9fcf-3794af014c80 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:39.965487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.987868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.988933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.989548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.990035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:39.990931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:39.991558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:39.992644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:39.994689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:39.998377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:39.999131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:39.999315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:39.999561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.000192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:40.000497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.001859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.011838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 328/656] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:40.012856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.023079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 329/657] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:40.024116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.143710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414f232-7a23-4289-ac6e-9b3de295fb25 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.157515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2295cb5-7d39-4fd4-9163-46df35020a81 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.190386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.191024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.191311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.191591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.192473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:40.192786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.194864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.205817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 329/658] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:40.205964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.205964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.205964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.205964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.207378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.215156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.216992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:40.229630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:40.230622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:40.231547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:40.235673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 330/659] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:40.236540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.275315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.276296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.323176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59e6a77-d9d3-4d09-8ec7-b042f9172f44 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.362985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9283106-0326-49bd-9e5b-99546bc482de None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.373030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.373894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.374326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.375138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.376029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] RBAC: Adding query filter params (interface=public, service_id=9b316f35d3544758a16ba7968dc42862, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:40.376786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.378686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.386200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 330/660] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/endpoints?service_id=9b316f35d3544758a16ba7968dc42862&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:40.391818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.407762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.408998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.409458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.409902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.410837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:40.411352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.412970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.436617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 331/661] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:40.437764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.504680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db893b8c-3df8-44c6-90ca-18d6629e9ebe None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.529954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.537420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.543026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.543660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.543912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.544223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.544779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.546143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903d8e46-7dec-4d00-ae7c-9a85f6c6e169 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.546306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c105f4-671a-4338-8561-400f4f517ca0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 331/662] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/services => generated 1029 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30954-f2c2-41bd-857a-533438d0fbf9 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30954-f2c2-41bd-857a-533438d0fbf9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d30954-f2c2-41bd-857a-533438d0fbf9 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 332/663] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.562484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.563155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.581465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:40.582596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:40.597362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.598222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.598638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.599009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.610852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.612540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:40.634803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:40.636045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:40.637218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:40.637218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:40.637218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:40.643301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 332/664] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:40.644413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.682972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 333/665] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:40 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:40.773882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2aac2c-64e1-46d8-8118-91b891deae84 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.810584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.811159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.811426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.811700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.812342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:40.812645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.814082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:40.827418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 333/666] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:40.832864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.936549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988dc3e7-ad30-4ff2-8455-d0694a68420a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:40.954397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.969335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:40.975097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:40.975993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:40.976674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:40.977155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:40.979596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:40.988229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:40.990039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:41.004698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:41.005653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:41.006579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:41.010002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 334/667] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:41.018498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:41.092900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9482c34-bac0-4713-8bc6-88b6b42196b1 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:41.132664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:41.133476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.133760 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.134027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.134741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:41.136104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:41.146668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6422081c-0fd5-44ee-b9a0-373b3aa630e8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:41.155927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 334/668] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:40 2022] POST /identity/v3/services => generated 241 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:59:41.201011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:41.202112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.202635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.203119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.204066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:41.204581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:41.206387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc285e4-16f1-41bf-a57c-e5d133df9faa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:41.223004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 335/669] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:41.229583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:41.308598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:41.355705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed57ce3a-43f3-4110-ba37-05e234b95996 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:41.393260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:41.393931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.394216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.394524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/768efb731b82485ba4e0a51901710a16/roles/02f2f600d68643b7b66c2d199f560efe` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.396684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=768efb731b82485ba4e0a51901710a16, role_id=02f2f600d68643b7b66c2d199f560efe)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:41.398119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd079ed-e732-4925-96bd-baff38c5a151 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:41.413823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 335/670] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 01:59:41 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/768efb731b82485ba4e0a51901710a16/roles/02f2f600d68643b7b66c2d199f560efe => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 01:59:41.446847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5587dea-74b1-4619-befb-40d65964cecc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.447423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5587dea-74b1-4619-befb-40d65964cecc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.447423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5587dea-74b1-4619-befb-40d65964cecc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.449115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 336/671] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:41.455199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadf247-5cbe-49c2-b5cc-c3d002dfcd01 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.455477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadf247-5cbe-49c2-b5cc-c3d002dfcd01 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.455679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffadf247-5cbe-49c2-b5cc-c3d002dfcd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.475629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ffadf247-5cbe-49c2-b5cc-c3d002dfcd01 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:41.477706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ffadf247-5cbe-49c2-b5cc-c3d002dfcd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:41.556750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 336/672] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:41 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:41.827246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5587dea-74b1-4619-befb-40d65964cecc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:41.863794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:41.936510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5587dea-74b1-4619-befb-40d65964cecc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:41.972460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:41.973071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:41.973354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:41.973640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:41.974301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:41.975770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b297bae0-ad0a-40c1-a63e-0070edcd07b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:41.991754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 337/673] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:41 2022] POST /identity/v3/services => generated 251 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:42.997169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f28b7f-62b8-47cd-9039-3e1480191a34 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:42.997169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f28b7f-62b8-47cd-9039-3e1480191a34 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:42.997169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f28b7f-62b8-47cd-9039-3e1480191a34 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:42.997797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 337/674] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:43.006284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf1be3-4ac9-480c-b8ea-2de419819831 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.006284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf1be3-4ac9-480c-b8ea-2de419819831 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.006284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4daf1be3-4ac9-480c-b8ea-2de419819831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.023350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4daf1be3-4ac9-480c-b8ea-2de419819831 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:43.025077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4daf1be3-4ac9-480c-b8ea-2de419819831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:43.107993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 338/675] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:43.382146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f28b7f-62b8-47cd-9039-3e1480191a34 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:43.446530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:43.495805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f28b7f-62b8-47cd-9039-3e1480191a34 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:43.504905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb801323-74cc-4b28-9eb4-914a51f933bb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.506823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb801323-74cc-4b28-9eb4-914a51f933bb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.506823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb801323-74cc-4b28-9eb4-914a51f933bb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.507025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 339/676] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:43.513719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.514129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.514555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.533902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:43.534585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:43.535391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:43.535542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.535956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.536364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.537084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:43.539794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:43.548841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:43.549965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Oct 23 01:59:43.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:43.553449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:43.553582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 338/677] 10.222.0.23 () {56 vars in 1132 bytes} [Sun Oct 23 01:59:43 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:43.559961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:43.623103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 340/678] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:43.678215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13a1c6-32b2-4c61-a21f-30b5bf66fcea None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:43.701533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:43.720074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:43.720960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.721238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.721513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.722150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:43.722464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:43.723873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cb3557-f1fb-4c03-95c8-56cd224eddc9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:43.730332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 339/679] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:43 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:43.736007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:43.814715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a709-bae9-48d8-ae8d-079f297f9868 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.815092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a709-bae9-48d8-ae8d-079f297f9868 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.815436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5a709-bae9-48d8-ae8d-079f297f9868 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.818088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 340/680] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:43.822697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.823044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.823406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.841729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:43.843005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:43.843638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a014cd28-857e-4590-82c6-61ced71cfa06 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:43.887074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:43.888255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:43.888587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:43.888879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:43.889524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:43.889830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:43.892146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:43.898415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 341/681] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:43 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:43.904751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:43.922567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:43.923620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 341/682] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:43 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:43.924387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.044261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4eb5caa-23bb-4d1e-832f-3001c90de7cb None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.048970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69eaa59-466f-4071-8680-12b4a3ee50c7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.067188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.092368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.093008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.093293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.093568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.094235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.095896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.096912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.097773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.098320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.098815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.099767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.101654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:44.112011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:44.113040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:44.113875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:44.121246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 342/684] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:44.123960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.125839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 342/684] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:43 2022] POST /identity/v3/endpoints => generated 347 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:44.195480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.248559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937bbda-74b3-4354-ab6b-76a52c4ac7e4 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.287714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.306882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.308109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.308392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.308964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.309622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:44.309931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.312256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.319531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 343/685] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:44.325674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.338522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8acd25-cadc-427d-87ff-023a7ea4ec79 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.377894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.379926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.380664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.381041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.381377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.396938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 01:59:44.403199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:44.404185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:44.405049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 343/686] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:44.410174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.453852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c9032d-acdc-4ce8-8e9f-961ee1073dca None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.489301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.490020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.490302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.490609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.491234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:44.491539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.493140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278941bf-a7f1-434e-8f04-f74a5727772d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.498388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 344/687] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:44.499415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9ba3-c514-4015-ab94-d7b365cfeea6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.499700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9ba3-c514-4015-ab94-d7b365cfeea6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.499973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbb9ba3-c514-4015-ab94-d7b365cfeea6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.501301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 345/688] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:44 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:44.503810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9ba3-c514-4015-ab94-d7b365cfeea6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.528917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c86f30a-065b-4326-b696-1bfa7cb4ed97 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.558368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.566377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.567095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.567382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.567656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.568315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:44.569174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.571376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e26a2-c022-47d6-9520-dc8c119e9e01 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.584155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 344/689] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:44.584155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.584155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.584155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.601094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:44.602401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:44.623464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbb9ba3-c514-4015-ab94-d7b365cfeea6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.625120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.662800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.663571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.664045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.664318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.665126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] RBAC: Adding query filter params (interface=public, service_id=5326a7f5c70b4c2491925f1ff69d8c19, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:44.665443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.667016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.671797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 345/690] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:44 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:44.672667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.688250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 346/691] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/endpoints?service_id=5326a7f5c70b4c2491925f1ff69d8c19&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:44.689423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.802667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86282ef-703a-476e-8450-c1ed00fcf7cf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.826164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfb4a5e-355f-44c8-bdb4-8d1eff0490bd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:44.830533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.846155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.846879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.847162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.847442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.860056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.862062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.871652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:44.872672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:44.873055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:44.873393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:44.874108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:44.875569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:44.877880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:44.878848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:44.879708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:44.879708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:44.879708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:44.882574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 347/692] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/services => generated 1497 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:59:44.884320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 346/693] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:44.885550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:44.922980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:44.948091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.010168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967c70e7-376c-414b-b26c-2ca48190c3e0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.060365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.061030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.061261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.061524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.062176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:45.063142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.064859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.076392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11a5378-17f1-4dc2-820d-eb9bb94507e6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.079407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 348/694] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:45.085329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.116350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.117036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.117327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.117604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.118191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.119638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:45.128546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:45.129550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:45.130370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 347/695] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:45.136138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.188712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.194655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.205685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f7eeb1-bf3a-4075-b577-b79206e2b6f5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.241621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.242461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.242775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.243050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.250578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.252031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:45.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:45.266487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Oct 23 01:59:45.268696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:45.269959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277aa1bd-4ed5-47b4-aba4-6e25fda95ac8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.278534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 349/696] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:45.278534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.311604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.312368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.313275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.313692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.314431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:45.315291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.317509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.326594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 348/697] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:45.330594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.386253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d754b-59d9-43dd-b076-7c6142bfc32a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.424525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.425409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.425792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.426169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.427168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:45.427686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.429203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.431504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c1d17-f70d-4e9e-a319-fbe6518bcd87 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.434055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.455547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 350/698] 10.222.0.23 () {56 vars in 1149 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:45.461715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.475615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.481357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.481996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.482266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.482659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.483538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:45.483844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.485223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.490533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 349/699] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:45.496529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.581540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e0806-aa6f-4657-a5db-a3024bc28198 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.615283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346694e8-2d42-4e95-9f3a-059eca60f000 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.623750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.624366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.624652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.624927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.659970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.660969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.661326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.661949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.663159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] RBAC: Adding query filter params (interface=public, service_id=01fad393a5ef475aa98376045bb8ac6e, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:45.663902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.665983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.674609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 350/700] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/endpoints?service_id=01fad393a5ef475aa98376045bb8ac6e&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:45.678337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.717789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.717946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.721418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.733337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:45.798821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aae1ce-b296-4826-886c-6d38a3afa2cd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.832673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 351/701] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Oct 23 01:59:45.834909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:45.835536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:45.835851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:45.836130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:45.836956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:45.838012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:45.838210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:45.850469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 351/702] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/services => generated 1497 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:59:45.960974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8226bfb6-c8ea-4e04-9469-75a054dd7353 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:45.973029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:46.003109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:46.004233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.004739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.005218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.017736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:46.019634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 01:59:46.036563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:46.037554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 01:59:46.038508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 01:59:46.038508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 01:59:46.038508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:46.041290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 352/703] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:46.046869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:46.154751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:46.163476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777fc0f1-683d-4083-88b3-e0239ecb6cbc None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:46.203894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:46.204891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.205397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.205873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.206755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:46.207279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:46.208842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:46.221861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 352/704] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 01:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:46.227555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:46.331292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d1a1df-5da4-4e86-85a1-c4178fe36ce9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:46.368871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:46.369562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.369844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.370117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.377177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:46.378786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5d195845-e063-473a-8a8b-3f5d5a19012c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:46.390314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:46.392009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:46.392909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 353/705] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:46.397621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:46.508088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5ee7b-0ede-4c6c-82c9-18e7fc3f4b5d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:46.542786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:46.543401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.543682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.543956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.544649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:46.544945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:46.546370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:46.563073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 353/706] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 01:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:46.564758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ab14-01bd-48ce-9ec6-fb892b35eb34 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.565057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ab14-01bd-48ce-9ec6-fb892b35eb34 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.565336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ab14-01bd-48ce-9ec6-fb892b35eb34 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.565899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:46.566959 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 354/707] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:46.572229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.572518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.572801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.582945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:46.593521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:46.595698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:46.615190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:46.682537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2fe11-82d1-4f29-83ff-1ff654f4e92d None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:46.694591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 355/708] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:46 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:46.729217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:46.729828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:46.730113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:46.730391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:46.731142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] RBAC: Adding query filter params (role.id=02f2f600d68643b7b66c2d199f560efe, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=768efb731b82485ba4e0a51901710a16) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:46.731453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:46.732959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:46.739517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 354/709] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 01:59:46 2022] GET /identity/v3/role_assignments?user.id=768efb731b82485ba4e0a51901710a16&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=02f2f600d68643b7b66c2d199f560efe => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:46.972841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:47.077267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce6cfc6-60a8-4500-b327-daf7edd21fe9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:47.113772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:47.114818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.115324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.115798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.116578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:47.118202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:47.127505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:47.128508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:47.129392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.132124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 356/710] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:47.138274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:47.220843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:47.250317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d2f94-2803-4804-b020-ba1c0d748eab None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:47.289785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:47.290472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.290761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.291033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.291654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:47.291954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:47.293371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a9a10-a00e-4964-a9fa-86325244a7bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:47.298610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 355/711] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:47 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:47.307793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:47.325784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:47.379395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca6c2-74c8-4835-8c36-5910a1c4d3ee None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.379760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca6c2-74c8-4835-8c36-5910a1c4d3ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.379964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cca6c2-74c8-4835-8c36-5910a1c4d3ee None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.381458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 356/712] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:47.387486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.387972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.388347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.408954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:47.410402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:47.430327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4688df-8fda-42b2-a8f1-58d1db354e4f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:47.475825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:47.477261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.477729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.478391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.479625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:47.480346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:47.482171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:47.488170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 357/713] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:47 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:47.488437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 357/714] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:47 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:47.759008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:47.841113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:47.863927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2680e8e8-fff0-4a04-a97b-0e360c1349d0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:47.899719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:47.900352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:47.900635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:47.900910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:47.901496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:47.902899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:47.911743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:47.912708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:47.913537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 358/715] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:47.919174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:47.994015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:48.025620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca85dac4-a436-4031-8513-1b246508b2ad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:48.064413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:48.065041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.065322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.065597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.066244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:48.066581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:48.067982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19c2c06-c20a-498c-8b91-6fb09aa947bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:48.074290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 358/716] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:48.080015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:48.180821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce64259-26e3-4fd4-b9ce-06a630762179 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.181107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce64259-26e3-4fd4-b9ce-06a630762179 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.181387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce64259-26e3-4fd4-b9ce-06a630762179 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.183052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 359/717] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:48.184920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e4f3d-423f-4dc6-870b-1b577d89c6cd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:48.188546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.189075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.189586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.208891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:48.210683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:48.224317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:48.225200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.225496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.225775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.227111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:48.227428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:48.228983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:48.235515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 359/718] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:48.241129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:48.251819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:48.302743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 360/719] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:48.347791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2f26c3-a879-4845-af64-23a2acd08d4e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:48.381764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:48.382397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.383208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.383482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.384147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:48.385508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:48.412436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 360/720] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:48 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:48.596844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:48.667440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:48.705203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aba6b67-bd21-4841-aa08-07c82c42c8c9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:48.746010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:48.746679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.746963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.747237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.747814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:48.749199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:48.758238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:48.759263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Oct 23 01:59:48.760117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:48.764419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 361/721] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:48.765534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:48.834790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:48.883297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f48cc-a0a3-4f17-87b0-27e3f9fc9595 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:48.919497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:48.920108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.920389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.920725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.921353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:48.921709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:48.923232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de8d52-2b7b-4a4c-a13f-613f0b8fe200 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:48.928378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 361/722] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:48 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:48.934121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:48.979056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd33e03-7842-42c1-bd6a-074e2424a39d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.979368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd33e03-7842-42c1-bd6a-074e2424a39d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.979571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd33e03-7842-42c1-bd6a-074e2424a39d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:48.980876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 362/723] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:48.986000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:48.986286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:48.986622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.005784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:49.006992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:49.055580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbe45d9-d090-44ab-b21f-0497141fe339 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:49.080703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 363/724] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:48 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:49.087232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:49.097015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:49.098118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.098653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.099150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.100050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:49.100555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:49.102772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:49.108721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 362/725] 10.222.0.23 () {56 vars in 1114 bytes} [Sun Oct 23 01:59:48 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:49.114720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:49.174859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:49.226082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcdc767-1a0b-4d5e-97ee-47c691af692f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:49.262428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:49.263067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.263347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.263625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.264303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:49.265729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:49.290759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 364/726] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:49 2022] POST /identity/v3/endpoints => generated 346 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:49.356507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:49.467036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b70e575-200a-43c2-ba36-7be1d397d471 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:49.509308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:49.509944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.510221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.510520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.511087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:49.512443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:49.521323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:49.522296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:49.523921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:49.526531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 363/727] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Oct 23 01:59:49 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:49.529170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:49.649011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123ecf96-9375-4fd8-9958-ec8ae77677b8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:49.665241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:49.689781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:49.690402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.690715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.690983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.691618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:49.691925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:49.693356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33776c98-2500-4baf-a07d-0552cd1957d7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:49.698775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 365/728] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:49.704819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:49.760566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:49.774140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab9f1-15a7-42b5-824b-7c967bde6bf0 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.774430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab9f1-15a7-42b5-824b-7c967bde6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.774835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08eab9f1-15a7-42b5-824b-7c967bde6bf0 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.776339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 366/729] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:49.782168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.782466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.782781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.801366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:49.802601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:49.822776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ce8c7b-2723-4b0d-a511-5747990dab49 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:49.861312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:49.862310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:49.862913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:49.863481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:49.864299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:49.864798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:49.866574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82422e35-f9e4-4eee-afe7-603107159155 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:49.878592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 364/731] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:49 2022] GET /identity/v3/services?type=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:49.878853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 367/731] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:49 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:50.147092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:50.252452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06270be-4884-4899-9762-d238e1ccd531 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:50.260689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:50.288801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:50.289407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:50.289689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:50.289968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:50.290653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:50.292017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920995a4-65a8-43fe-b791-ee510c45b7df None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:50.309335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 368/732] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:50 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:50.736492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75655748-0a52-476f-8aa3-e118c4f27c8b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:50.736995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75655748-0a52-476f-8aa3-e118c4f27c8b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:50.737071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75655748-0a52-476f-8aa3-e118c4f27c8b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:50.738683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 365/733] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:50.744022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:50.744438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:50.744628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:50.762948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:50.764128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:50.846088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 369/734] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:50 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:51.116316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75655748-0a52-476f-8aa3-e118c4f27c8b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:51.228469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75655748-0a52-476f-8aa3-e118c4f27c8b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:51.243906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:51.275028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:51.276082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.276558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.277042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.277860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:51.279478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:51.290112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:51.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:51.291992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:51.294703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 366/735] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:51.302495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:51.397933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:51.407012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7bb37-e330-42a0-a00a-c57583ab018f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:51.442886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:51.443544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.443828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.444099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.444720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:51.445200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:51.446632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:51.451859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 370/736] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:51.457160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:51.561678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2749ce90-2276-45ea-9e87-cd053b8aedf6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:51.600435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:51.601224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.601621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.602267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.602567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:51.602945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:51.604471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a70c1fa-ddbb-43dc-af4c-8a947ee2fd30 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:51.610598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 367/737] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:51.616629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:51.708878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b211013-9bb4-4f79-ab9f-eb97f50191d6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.709168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b211013-9bb4-4f79-ab9f-eb97f50191d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.709450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b211013-9bb4-4f79-ab9f-eb97f50191d6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.711383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 368/738] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:51.719203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.719624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.720670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.729654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748f2ac-e671-4365-984a-154d31a11188 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:51.741206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:51.743861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:51.769975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:51.771388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.771873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.772360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.773631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] RBAC: Adding query filter params (interface=admin, service_id=5326a7f5c70b4c2491925f1ff69d8c19, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:51.774251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:51.775890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:51.786759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 371/739] 10.222.0.23 () {56 vars in 1249 bytes} [Sun Oct 23 01:59:51 2022] GET /identity/v3/endpoints?service_id=5326a7f5c70b4c2491925f1ff69d8c19&interface=admin®ion_id=RegionOne => generated 193 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:51.788266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:51.790164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:51.817261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:51.843061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 369/740] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:51 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:51.906145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4268c268-0e01-4289-ab99-c6f5bd408d44 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:51.940752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:51.941401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:51.941685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:51.941957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:51.942586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:51.944014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63f40a-31bc-4f94-8b88-b6c707c7bae5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:51.951404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 372/741] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:51 2022] GET /identity/v3/services => generated 1727 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:59:52.141381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:52.198609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ec6e-6da2-4dcc-9e41-c6c53f101577 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.198931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ec6e-6da2-4dcc-9e41-c6c53f101577 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.199152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1ec6e-6da2-4dcc-9e41-c6c53f101577 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.200733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 373/742] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:52.205635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.205857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.206132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.225595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:52.227598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:52.248391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3654fb-858e-4f80-8b5b-b02b9dd04c6a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:52.287147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38898722-0b00-46eb-b888-32093962d978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:52.287752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38898722-0b00-46eb-b888-32093962d978 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.288028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38898722-0b00-46eb-b888-32093962d978 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.288294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38898722-0b00-46eb-b888-32093962d978 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.289564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38898722-0b00-46eb-b888-32093962d978 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:52.291013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38898722-0b00-46eb-b888-32093962d978 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:52.293595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING keystone.server.flask.common [None req-38898722-0b00-46eb-b888-32093962d978 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 01:59:52.306201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 374/743] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:52 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:52.319336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.sql.core [None req-38898722-0b00-46eb-b888-32093962d978 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 01:59:52.319336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 01:59:52.319336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [parameters: {'id': 'bf44a5c7c6134a669a75fabd72430b2a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 01:59:52.319336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81625) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-38898722-0b00-46eb-b888-32093962d978 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf44a5c7c6134a669a75fabd72430b2a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf44a5c7c6134a669a75fabd72430b2a. Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:52.327293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 01:59:52.328146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application yield resource Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise self.value Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application session.commit() Oct 23 01:59:52.329115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.session.flush() Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application compat.raise_( Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 01:59:52.330010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 01:59:52.330963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:52.331790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [parameters: {'id': 'bf44a5c7c6134a669a75fabd72430b2a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:52.332654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:52.333439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bf44a5c7c6134a669a75fabd72430b2a. Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:52.336793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 370/744] 10.222.0.23 () {60 vars in 1140 bytes} [Sun Oct 23 01:59:52 2022] POST /identity/v3/projects => generated 262 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:52.337448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:52.359712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:52.448923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b567bf-9a29-46d3-ac97-a55148580bc4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:52.485910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:52.486486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.486774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.487046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.495265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:52.496774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 01:59:52.509787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:52.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 01:59:52.514227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:52.515433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 375/745] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:52.520163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38898722-0b00-46eb-b888-32093962d978 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:52.551052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:52.588356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:52.637607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38898722-0b00-46eb-b888-32093962d978 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:52.668891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:52.690245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:52.691074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.691367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.691957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.692911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:52.693220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:52.694336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f150f155-670c-4626-951a-8a3048975ce9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:52.695533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f5448-fdc1-4483-a583-a306c5d1360d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:52.719016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 371/746] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:52 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:52.720581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbb59-6575-405c-a118-f686b918eaf0 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.720864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbb59-6575-405c-a118-f686b918eaf0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.721141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecbb59-6575-405c-a118-f686b918eaf0 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.722548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 372/747] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:52.732710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.733067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.733267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.739658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:52.740636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:52.741125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:52.741674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:52.742577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:52.744396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:52.756464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:52.759582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:52.770911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 376/748] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:52 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 01:59:52.856137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 373/749] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:52 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:53.128423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:53.237287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9edffe-5011-43df-90dc-3a73ec1963aa None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:53.272608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:53.273804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:53.274083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:53.274411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:53.274962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:53.276346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:53.278126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:53.285918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:53.286989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:53.287845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 377/750] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:53 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:53.293356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:53.309152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:53.396150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ff159d-0ece-4aeb-855c-84a964ef4336 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:53.430755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:53.431362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:53.431644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:53.431918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:53.432539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:53.432842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:53.434208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:53.439435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 374/751] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:53 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:53.445705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:53.549607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818b59e8-75a5-41f2-8082-52e6deab6a65 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:53.584853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:53.585586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:53.585885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:53.586129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:53.586783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:53.587086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:53.588458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:53.593650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 378/752] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:53 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:53.599723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:53.680623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:53.702694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87cfcfcc-6f41-4c2a-9fa6-4f79519e32d8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:53.740425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:53.741034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:53.741315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:53.742153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:53.742992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] RBAC: Adding query filter params (interface=public, service_id=5c118cc9ee2d4fce82739951f3953957, region_id=RegionOne) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:53.743404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:53.744772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ce5a74-a81f-47fe-964e-443e295312bc None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:53.750404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 375/753] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:53 2022] GET /identity/v3/endpoints?service_id=5c118cc9ee2d4fce82739951f3953957&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:53.756021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:53.826878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:53.867795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68205dc2-6807-4b24-89ab-62d89a799d48 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:53.902482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:53.903089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:53.903369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:53.903641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:53.904235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:53.905604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8889a6-ae68-45d2-8b4b-1c94533591cf None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:53.912537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 379/754] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:53 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 01:59:54.238240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065bb50-c42b-40fd-b294-8be942107344 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:54.238796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065bb50-c42b-40fd-b294-8be942107344 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:54.238796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1065bb50-c42b-40fd-b294-8be942107344 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:54.240425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 376/755] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:54.245790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:54.246081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:54.246359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:54.265424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:54.266631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:54.336145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 380/756] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:54 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:54.624157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065bb50-c42b-40fd-b294-8be942107344 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:54.730897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1065bb50-c42b-40fd-b294-8be942107344 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:54.765810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:54.766466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:54.766744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:54.767077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:54.767653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:54.769071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:54.778721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:54.779693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:54.780538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 377/757] 10.222.0.23 () {56 vars in 1126 bytes} [Sun Oct 23 01:59:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:54.786209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:54.847057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:54.898679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e3cc72-687d-473b-a644-9ae90ffb8c46 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:54.937700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:54.938326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:54.938636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:54.938911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:54.939557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:54.939861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:54.941298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359ea405-fc02-4773-9957-955d0416d7ec None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:54.948287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 381/758] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:54.953901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:54.966780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.041880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c155-796e-492d-ae77-4063ebb39993 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.042148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c155-796e-492d-ae77-4063ebb39993 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.042430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc14c155-796e-492d-ae77-4063ebb39993 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.046564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 382/759] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:55.049127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4dd2-58ca-41a4-90d3-6a1c75955edf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.049487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4dd2-58ca-41a4-90d3-6a1c75955edf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.049768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37c4dd2-58ca-41a4-90d3-6a1c75955edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.067895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c37c4dd2-58ca-41a4-90d3-6a1c75955edf None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:55.069071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c37c4dd2-58ca-41a4-90d3-6a1c75955edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:55.070235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1eefa0-b5d8-43fa-b1ef-f086110eff77 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.115437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.116459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.116805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.117081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.118001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:55.118308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.120499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d266f8b-9abe-47f9-af47-b2791c03882d None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.126417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 378/760] 10.222.0.23 () {56 vars in 1122 bytes} [Sun Oct 23 01:59:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:55.128040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c8d5-f4ff-460f-9ec7-fa2e110e2831 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.128250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c8d5-f4ff-460f-9ec7-fa2e110e2831 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.128526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd5c8d5-f4ff-460f-9ec7-fa2e110e2831 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.129843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 379/761] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:55.134934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c8d5-f4ff-460f-9ec7-fa2e110e2831 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.141525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 383/762] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:55.144246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.144768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.145276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.171110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:55.173708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:55.208500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.244077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd5c8d5-f4ff-460f-9ec7-fa2e110e2831 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.248704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.260979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 384/763] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:55 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:55.290295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.291288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.291799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.292221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.293073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.294597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.313391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 380/764] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:55 2022] POST /identity/v3/endpoints => generated 325 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:55.421241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.531026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b28de-13ee-48f8-a389-f8a081820a4e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.535334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.574247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.574844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.575124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.575399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.576024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.577458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.580038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING keystone.server.flask.common [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Oct 23 01:59:55.595640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.sql.core [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 01:59:55.595640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 01:59:55.595640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [parameters: {'user_id': '20c771ee8d2b4983a06a47d0a95585b4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 01:59:55.595640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:55.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:55.620290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application yield resource Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise self.value Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application session.commit() Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 01:59:55.621285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.flush() Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application compat.raise_( Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 01:59:55.622169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = connection._execute_20( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 01:59:55.623084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 01:59:55.623936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [parameters: {'user_id': '20c771ee8d2b4983a06a47d0a95585b4', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:55.624747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Oct 23 01:59:55.625553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.626626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 385/765] 10.222.0.23 () {60 vars in 1131 bytes} [Sun Oct 23 01:59:55 2022] POST /identity/v3/users => generated 160 bytes in 200 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:55.628630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.668146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e163b4b-ed7a-436b-bed6-43e32eee5855 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.674279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.707270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.716249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.717088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.717510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.718035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.718913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.720624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:55.730833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:55.731878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:55.732721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 381/766] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Oct 23 01:59:55 2022] GET /identity/v3/services/image => generated 86 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 01:59:55.737613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.761826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac6599-e45d-4b6f-8162-7f80388b584e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.809713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.810350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.810660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.810934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.818958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.821183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 01:59:55.834159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:55.835576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 01:59:55.836902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 386/767] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 01:59:55 2022] GET /identity/v3/users/cinder => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:55.843488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.856471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4534339d-d173-4af8-af49-2344c3bae95a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.901286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:55.902282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:55.902658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:55.902995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:55.904037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] RBAC: Adding query filter params (name=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:55.904349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:55.906596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:55.914820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 382/768] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:55.920397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:55.974781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:55.981649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed013002-2865-4176-95ae-e71014067e7b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:55.986999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:56.021522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783890-f684-416b-899f-3364e3996057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.022496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783890-f684-416b-899f-3364e3996057 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.023850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783890-f684-416b-899f-3364e3996057 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.024325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67783890-f684-416b-899f-3364e3996057 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.025305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783890-f684-416b-899f-3364e3996057 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:56.025857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783890-f684-416b-899f-3364e3996057 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.027794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67783890-f684-416b-899f-3364e3996057 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.030321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90263bc0-a76a-4c4d-b239-566fc39c227c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:56.067364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.067972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.068251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.068517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.069137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:56.069437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.071457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 387/769] 10.222.0.23 () {56 vars in 1101 bytes} [Sun Oct 23 01:59:55 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:56.074739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a318c753-e938-4ef8-8729-6da2db486520 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.083339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 383/770] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 01:59:56.090378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783890-f684-416b-899f-3364e3996057 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:56.213991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67783890-f684-416b-899f-3364e3996057 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:56.254515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.255493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.255978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.256462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.257709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] RBAC: Adding query filter params (interface=public, service_id=9dbff7c2a8e146c28004a1fe1686c917, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:56.258378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.260188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.260610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772694-80fc-42dc-ac12-5c86affca7a4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.261035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772694-80fc-42dc-ac12-5c86affca7a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.261192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94772694-80fc-42dc-ac12-5c86affca7a4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.262654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 384/771] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:56.266276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 388/772] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 01:59:56 2022] GET /identity/v3/endpoints?service_id=9dbff7c2a8e146c28004a1fe1686c917&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:56.268396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.268676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.268949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.271783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:56.290618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:56.292164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:56.296667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:56.381759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 385/773] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:56 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:56.396498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f0106-59c0-4941-843c-fc0d510ebd39 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:56.435690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.436306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.436589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.436874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.437478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.439028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.446080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 389/774] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 01:59:56 2022] GET /identity/v3/services => generated 1957 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 01:59:56.667852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:56.724090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:56.771786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7e510-804c-4cbc-ac64-1356db37f47c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:56.806093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.806749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.807026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.807296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.807870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.809227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:56.817809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:56.819010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Oct 23 01:59:56.819876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:56.822516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 386/775] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 01:59:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:56.825357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:56.908046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:56.925822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02097182-171c-493e-bc75-dc02b123ae17 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:56.959978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:56.961361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:56.961605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:56.961879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:56.962526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:56.962832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:56.964213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:56.973967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 390/776] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:56.975518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:57.077091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822cf0fe-c3c9-4092-a2bc-ebbdeb5233b5 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:57.092787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:57.112118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:57.112724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:57.113000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:57.113271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:57.113889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:57.114188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:57.115588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879baeae-42bd-46c9-a1e5-373040d21647 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:57.125767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 387/777] 10.222.0.23 () {56 vars in 1112 bytes} [Sun Oct 23 01:59:56 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:57.126906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:57.227688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17a2e12-5d0a-495e-bccb-75a7e40deae2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:57.261590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:57.262192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:57.262486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:57.262759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:57.263414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:57.264772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d19011f-d459-4e7b-96be-deef2cb47c68 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:57.290490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 391/778] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:57 2022] POST /identity/v3/endpoints => generated 334 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:57.635966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc72e7-94e2-462e-93d9-4350a37213fc None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:57.636478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc72e7-94e2-462e-93d9-4350a37213fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:57.636555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc72e7-94e2-462e-93d9-4350a37213fc None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:57.638033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 388/779] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:57.643360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:57.643715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:57.644061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:57.652042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:57.665041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:57.666820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:57.744686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 392/780] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:58.012463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc72e7-94e2-462e-93d9-4350a37213fc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:58.116227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc72e7-94e2-462e-93d9-4350a37213fc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:58.151534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:58.152165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:58.152448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:58.152723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:58.153834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:58.155301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7104479-3285-4ceb-9a2b-d738f1780244 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:58.157041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 389/781] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:58.162624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:58.265030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600ae22e-e10d-482a-8e91-930521650e7a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:58.294064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:58.301210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:58.301820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:58.302102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:58.302374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:58.303047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:58.305071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8cf5ee-d6e0-4b44-9e9b-1a128bdf9cf7 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:58.332886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 393/782] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 01:59:58 2022] POST /identity/v3/projects => generated 311 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:58.661059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347152f-9156-4961-a423-14a69cb024ca None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:58.661574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347152f-9156-4961-a423-14a69cb024ca None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:58.661652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347152f-9156-4961-a423-14a69cb024ca None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:58.663128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 390/783] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 01:59:58.668833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:58.669124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:58.669402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:58.689835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:58.691068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:58.759792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 394/784] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 01:59:59.028606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347152f-9156-4961-a423-14a69cb024ca None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:59.080437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:59.134201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347152f-9156-4961-a423-14a69cb024ca None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:59.167801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:59.168438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.168717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.168987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] PATH_INFO: `/v3/services/image` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.169562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:59.170960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 01:59:59.180572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 01:59:59.181571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Oct 23 01:59:59.182419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 01:59:59.183835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 391/785] 10.222.0.23 () {56 vars in 1105 bytes} [Sun Oct 23 01:59:59 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 01:59:59.187539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:59.288815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e78fe9-bb2b-4b59-9ace-f1831c4804b4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:59.301937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:59.325907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:59.326534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.326818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.327091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.327708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] RBAC: Adding query filter params (name=image) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:59.328010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:59.329382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:59.334328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 395/786] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:59.340153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:59.450355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7c5cac-1cfb-425f-8137-a9803acb92de None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:59.465289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 01:59:59.489502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:59.490108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.490389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.490690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.491315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] RBAC: Adding query filter params (type=image) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 01:59:59.492299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:59.493723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595dadbf-862e-4efa-9286-d1f6073968ec None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:59.503372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 392/787] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 01:59:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 01:59:59.504468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 01:59:59.544158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325ba7-9fda-4a0d-ad9c-6088da86d70d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.544448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325ba7-9fda-4a0d-ad9c-6088da86d70d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.544641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5325ba7-9fda-4a0d-ad9c-6088da86d70d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.545993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 393/788] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 01:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 01:59:59.550945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.551236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.551514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.570847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 01:59:59.572137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 01:59:59.608616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95309cac-3a86-4426-976e-b0ccc892c5b9 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 01:59:59.642596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 01:59:59.643199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 01:59:59.643476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 01:59:59.643744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 01:59:59.644397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 01:59:59.645786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 01:59:59.650719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 394/789] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 01:59:59 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 01:59:59.674486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 396/790] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 01:59:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 01:59:59.915567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.015717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d9c90e-9ef6-489e-8116-ca25790fb91a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.023316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:00.057212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.057846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.058117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.058390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.068551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:00.074050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:00.075191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 02:00:00.076025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:00.077218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 395/791] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 01:59:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:00.081326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.113802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:00.186869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ea5620-856e-4160-a309-453ed5c9f19b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.219873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.220774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.220992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.221325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.221985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:00.222290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:00.223698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:00.233927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 397/792] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:00.239425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.344439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e9fa7f-2bf7-4e75-bc58-4b695eb295c3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.379407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.380034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.380314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.380588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.391904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:00.393362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-19ee51a4-ddcb-491c-9c8d-5c4c26c45174 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:00:00.408225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:00.409202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:00:00.410098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:00:00.410098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:00.410098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:00.410098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 396/793] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:00:00.416867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.518103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:00.526428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c318f7aa-f416-4f16-af44-91f689fe31f8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.557613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11a55-3469-4dca-8771-e30379d68774 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.558012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11a55-3469-4dca-8771-e30379d68774 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.558361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e11a55-3469-4dca-8771-e30379d68774 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.559900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 397/794] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:00.565521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.565914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.566230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.566582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.566863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.567134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.567338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.567770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:00.568071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:00.569457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:00.582369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 398/795] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:00.588166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.591914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:00.593162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:00.604408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:00.674502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 398/796] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 02:00:00.698214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0521f-039e-4e0b-a08a-10b57f498417 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.733872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.734517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.734797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.735069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.742891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:00.744328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:00:00.755189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:00.756106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:00.757103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 399/797] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:00:00.761983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.863527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1977e0-24a7-43b5-a1d5-43a9c902fb8d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:00.898056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:00.898688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:00.898973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:00.899249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:00.899970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:00.900278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:00.901705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:00.922981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 399/798] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:00.925451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.942520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:00.949362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:00.953340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:01.034726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6dd31-0f87-4302-a4e8-d3b91e50374d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.060631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09092202-41e3-4e68-a1c4-59173fd42eee None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.072091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.072901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.073330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.073748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.084743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.086380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.097020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.097650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.097935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.098211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.099357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.101328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:00:01.103144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:01.104215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:00:01.105296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:00:01.105296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:01.105296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:01.105500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 400/799] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:01.111189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:01.111583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 400/800] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:00:01.112546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:01.180509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:01.183332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:01.214617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9920ddab-44fb-47ac-b761-92109f15e151 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.223051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3623518-03e8-46e0-8d96-98fb6a5e3adb None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.250128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.250755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.251031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.251307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.251948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.253352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.260085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.260939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.261402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.261858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.262803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:01.263389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.265185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.278061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 401/801] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 02:00:01 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:00:01.279597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 401/802] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:00:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:01.285339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:01.398459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ae815b-5bf8-459a-a2bd-ad0a53b5eb43 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.426896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:01.438679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.439252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.439535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.439809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.447256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.448903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:00:01.461666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:01.462649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:01.463557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 402/803] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:00:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:01.469068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:01.573964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924ebf22-5172-42f3-92a4-997a8e05e21b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.600259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:01.609890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.610521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.610808 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.612439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.612439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:01.612439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.618487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a81a76-a1f8-4307-8e51-3f044076a0ce None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.629681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 402/804] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:00:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:01.636450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:01.746527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d63f4c-32a5-4294-946e-80eba27e7a81 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:01.781383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:01.782492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.783013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.783508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/02f2f600d68643b7b66c2d199f560efe` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.786025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=8060598d22754154a1c7af65a96aaac2, role_id=02f2f600d68643b7b66c2d199f560efe)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:01.787815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b51a5-c91a-4293-a9cc-c7885e7243aa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:01.804992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 403/805] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:00:01 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/8060598d22754154a1c7af65a96aaac2/roles/02f2f600d68643b7b66c2d199f560efe => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:00:01.912465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be89-83f0-48fd-9fc8-3b78f80ca627 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.912776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be89-83f0-48fd-9fc8-3b78f80ca627 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.913029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57be89-83f0-48fd-9fc8-3b78f80ca627 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.914626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 403/806] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:01.920276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ca23-ce27-4086-938f-629891131b9b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:01.920593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ca23-ce27-4086-938f-629891131b9b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:01.920851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0786ca23-ce27-4086-938f-629891131b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:01.941023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0786ca23-ce27-4086-938f-629891131b9b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:01.942248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0786ca23-ce27-4086-938f-629891131b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:01.988673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:02.016694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 404/807] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:02.293241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be89-83f0-48fd-9fc8-3b78f80ca627 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:02.400283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57be89-83f0-48fd-9fc8-3b78f80ca627 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:02.433260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:02.433862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:02.434144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:02.434421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:02.435111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:02.436475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:02.445990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.sql.core [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 02:00:02.445990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 02:00:02.445990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [parameters: {'id': '111936e916b84008861ad90f17e37d4b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 02:00:02.445990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81624) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 02:00:02.453220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return method(*args, **kwargs) Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return project_ref.to_dict() Oct 23 02:00:02.454235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application yield resource Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application next(self.gen) Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.rollback() Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.force_reraise() Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise self.value Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application session.commit() Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Oct 23 02:00:02.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._prepare_impl() Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.session.flush() Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._flush(objects) Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application compat.raise_( Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application flush_context.execute() Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rec.execute(self) Oct 23 02:00:02.456481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application _emit_insert_statements( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application c = connection._execute_20( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self._execute_context( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Oct 23 02:00:02.457611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self.dialect.do_execute( Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result = self._query(query) Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application conn.query(q) Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application result.read() Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Oct 23 02:00:02.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application packet.raise_for_error() Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [parameters: {'id': '111936e916b84008861ad90f17e37d4b', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Oct 23 02:00:02.459658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:02.461096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.Conflict( Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:02.462028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 404/808] 10.222.0.23 () {60 vars in 1137 bytes} [Sun Oct 23 02:00:02 2022] POST /identity/v3/domains => generated 190 bytes in 162 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:02.464472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786ca23-ce27-4086-938f-629891131b9b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:02.567895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0786ca23-ce27-4086-938f-629891131b9b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:02.605102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:02.605730 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:02.606009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:02.606282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:02.616900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:02.618130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-b4f5ccf7-55c1-4826-b152-1dbd84ab79ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:00:02.633041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:02.634009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:00:02.640080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:00:02.640080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:02.640080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:02.640080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 405/809] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:02.641858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:02.699494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:02.753840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afa9c83-571f-4c0b-aa6b-e65151886282 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:02.787243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:02.787846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:02.788714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:02.788995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:02.789717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:02.790024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:02.791433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2679c1f-38ed-433f-a816-18afd2e39549 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:02.802574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 405/810] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:03.503506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756daf6b-cae0-4ac5-9d42-eea50774f42c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:03.504054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756daf6b-cae0-4ac5-9d42-eea50774f42c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:03.504054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756daf6b-cae0-4ac5-9d42-eea50774f42c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:03.505413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 406/811] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:03.511349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a1f7-a588-45d9-bcf1-d14d7d96b32b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:03.511817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a1f7-a588-45d9-bcf1-d14d7d96b32b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:03.511918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6943a1f7-a588-45d9-bcf1-d14d7d96b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:03.529531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6943a1f7-a588-45d9-bcf1-d14d7d96b32b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:03.530721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6943a1f7-a588-45d9-bcf1-d14d7d96b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:03.596750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 406/812] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:03.862219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756daf6b-cae0-4ac5-9d42-eea50774f42c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:03.894012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:03.968978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756daf6b-cae0-4ac5-9d42-eea50774f42c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:03.985370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212310-82d7-46a6-b4d6-7c10154ecb03 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:03.985797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212310-82d7-46a6-b4d6-7c10154ecb03 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:03.986071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c212310-82d7-46a6-b4d6-7c10154ecb03 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:03.987427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 407/813] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:00:03.992926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:03.993211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:03.993483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.004506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.005133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.005411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.005683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.011229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:04.012412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:04.016071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:04.020746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:04.021808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:04.023401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 407/814] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:04.027731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.048309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.081570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 408/815] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 02:00:04.114053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.126642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeec7bb3-1d0e-4731-aeb8-4b0930d72dbd None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.159182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.159778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.160061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.160337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.160988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:04.161292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.162691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.171038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 408/816] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:04.176844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.274008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666d776-fb66-415f-be8e-0f7a242d43e7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.306636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.307245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.307523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.307794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] PATH_INFO: `/v3/users/cb5ae3e523ff4af2b52ee57943fc0331` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.320287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] RBAC: Authorizing `identity:get_user(user_id=cb5ae3e523ff4af2b52ee57943fc0331)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.321726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.323344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 409/817] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/users/cb5ae3e523ff4af2b52ee57943fc0331 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:04.329035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.356264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.383486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.431013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86813993-1c1d-4de1-8d1e-64c373946ba9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.452739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab856afd-7a24-41be-92f1-3863829458cd None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.463017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.463635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.463917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.464187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.465276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] RBAC: Authorizing `identity:get_project(project_id=c701720de080402db19dcfb42502520b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.466732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.468344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 409/818] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/projects/c701720de080402db19dcfb42502520b => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:04.474463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.487105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.487727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.488003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.488270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.493818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.499225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:04.503925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:04.504974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:04.506219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 410/819] 10.222.0.23 () {56 vars in 1120 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:00:04.511340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.576513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337c0b9-ee31-43f7-bb88-73d946dd842a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.600627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.611250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.611912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.612192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.612465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b/users/cb5ae3e523ff4af2b52ee57943fc0331/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.613479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c565dd2-fb06-432a-919a-ef61cdd26baf None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.614356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] RBAC: Authorizing `identity:create_grant(project_id=c701720de080402db19dcfb42502520b, user_id=cb5ae3e523ff4af2b52ee57943fc0331, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.615807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.628349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 410/820] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:00:04 2022] PUT /identity/v3/projects/c701720de080402db19dcfb42502520b/users/cb5ae3e523ff4af2b52ee57943fc0331/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:00:04.650206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.650857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.651138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.651414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.652092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:04.652400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.653824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.663275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 411/821] 10.222.0.23 () {56 vars in 1115 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:04.669326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.754757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:04.779724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab478cb-5a64-4b4e-8cce-88cab2474ea8 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.814456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.815099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.815381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.815654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.826832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.828281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:00:04.842563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:04.843580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:00:04.844837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:00:04.844837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:04.844837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:04.844837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 411/822] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:04.849635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:04.947806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d3f6a-23d7-41c8-a869-087d69931376 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:04.980644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:04.981243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:04.981524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:04.981798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:04.982422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:04.982754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:04.984125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:04.996182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 412/823] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:00:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:05.001830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:05.100272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40862826-f8ac-4ab1-9df0-16c55bafe79a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:05.127977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:05.134989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:05.135622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:05.135902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:05.136176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:05.143675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:05.145115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:00:05.156616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:05.157746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:05.158808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 412/824] 10.222.0.23 () {56 vars in 1093 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:05.164118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:05.176276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:05.260431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857c5674-cc4e-4aac-a4ac-085a7767091b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:05.294279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:05.294901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:05.295179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:05.295453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:05.296159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:05.296460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:05.297855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:05.314477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 413/825] 10.222.0.23 () {56 vars in 1133 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:05.320786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:05.419798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ec222e-9eeb-4914-b301-2635a08874c6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:05.452017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:05.452618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:05.452898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:05.453169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:05.510530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:05.521614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:05.523711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:05.619732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 413/826] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:00:05.625279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:05.696361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:05.730024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b8fe64-6bac-45a1-95af-7e964d009385 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:05.764098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:05.764735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:05.765079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:05.765424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:05.775249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:05.776690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:00:05.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:05.791916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:00:05.792822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:00:05.792822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:00:05.792822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:05.792822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 414/827] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:05.797694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:05.885485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:05.895564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ffb9a3-3a2d-4133-8a38-4dc6bf587fa6 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:05.927843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:05.928440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:05.929279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:05.929553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:05.930288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:05.930628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:05.931998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:05.942827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 414/828] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:05.949517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:06.045613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f359a1a0-bf74-49a0-a310-6342e4ec27d2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:06.064443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:06.078911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:06.079526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:06.079802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:06.080070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:06.086332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:06.087779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:00:06.098828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:06.099779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:06.101191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 415/829] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:00:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:06.106218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:06.203920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c128a33d-d875-4aab-98b8-2062cb178e0a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:06.237029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:06.237630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:06.237912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:06.238188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:06.238893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:06.239204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:06.240613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba25499-8dc8-45b0-8b35-3f9b61f5c109 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:06.252717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 415/830] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:00:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:06.259829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:06.358463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e9995-a8be-4d1b-8e05-3a1f4bfbb5a2 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:06.391182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:06.391888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:06.392168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:06.392437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:06.393144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] RBAC: Adding query filter params (role.id=02f2f600d68643b7b66c2d199f560efe, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=8060598d22754154a1c7af65a96aaac2) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:06.393453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:06.394902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a293ba-4e3d-4978-9857-47a030822210 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:06.400698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 416/831] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:00:06 2022] GET /identity/v3/role_assignments?user.id=8060598d22754154a1c7af65a96aaac2&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=02f2f600d68643b7b66c2d199f560efe => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:06.794714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bfdd-27b9-4b9b-84dd-d6cb60b655a6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:06.795218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bfdd-27b9-4b9b-84dd-d6cb60b655a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:06.795304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1881bfdd-27b9-4b9b-84dd-d6cb60b655a6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:06.796610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 416/832] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:06.802051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:06.802532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:06.802660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:06.821191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:06.822328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:06.863168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:06.890653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 417/833] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:07.159156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881bfdd-27b9-4b9b-84dd-d6cb60b655a6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:07.172224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:07.263616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1881bfdd-27b9-4b9b-84dd-d6cb60b655a6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:07.297013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:07.297628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:07.297905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:07.298173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:07.308331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:07.313061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:07.314125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:07.315020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 417/834] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:07.320435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:07.416891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03309c54-d39d-4785-832c-4561e8cc1ec2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:07.563076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:07.563673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:07.563954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:07.564229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:07.564880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:07.565186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:07.566589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:07.575047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 418/835] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:07.581171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:07.649664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:07.679448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0312214-4764-47ec-80a0-a4d94e91c3d3 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:07.712818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:07.713437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:07.713718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:07.713993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] PATH_INFO: `/v3/users/cb5ae3e523ff4af2b52ee57943fc0331` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:07.715077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] RBAC: Authorizing `identity:get_user(user_id=cb5ae3e523ff4af2b52ee57943fc0331)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:07.716518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:07.718077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 418/836] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:07 2022] GET /identity/v3/users/cb5ae3e523ff4af2b52ee57943fc0331 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:07.724224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:07.759137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:07.822249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544f89a1-f508-4e45-a3e3-d4c662430a66 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:07.855865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:07.856459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:07.856735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:07.857003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:07.921118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:07.922584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:07.954666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:08.017216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 419/837] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:00:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:00:08.023234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:08.121809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aba56ec-90e1-438a-8c86-32a4f2d0944b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:08.154537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:08.155148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:08.155424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:08.155694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:08.156784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] RBAC: Authorizing `identity:get_project(project_id=c701720de080402db19dcfb42502520b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:08.158190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f019c66f-b2d4-493d-b6e5-89178c1a2404 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:08.159814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 419/838] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:08 2022] GET /identity/v3/projects/c701720de080402db19dcfb42502520b => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:08.166079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:08.261215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18924279-da1c-4d53-8368-e5bc541ef4f5 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:08.293954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:08.294575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:08.294855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:08.295126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:08.295828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=c701720de080402db19dcfb42502520b, user.id=cb5ae3e523ff4af2b52ee57943fc0331) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:08.296133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:08.297537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c385-d247-445b-be79-0c1f932db915 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:08.300236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:08.303990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 420/839] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:00:08 2022] GET /identity/v3/role_assignments?user.id=cb5ae3e523ff4af2b52ee57943fc0331&scope.project.id=c701720de080402db19dcfb42502520b&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:10.480462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509dcee-a93d-45c6-b309-ded73ebcb030 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:10.480983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509dcee-a93d-45c6-b309-ded73ebcb030 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:10.480983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7509dcee-a93d-45c6-b309-ded73ebcb030 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:10.482328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 420/840] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:10.487520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:10.487927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:10.488082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:10.507520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:10.508396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:10.572839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 421/841] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:10.840482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509dcee-a93d-45c6-b309-ded73ebcb030 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:10.849441 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:10.940521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7509dcee-a93d-45c6-b309-ded73ebcb030 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:10.973627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:10.974237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:10.974539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:10.974811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:10.976645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:10.978060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bebd47-f21f-4498-8598-b6863a638e02 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:10.979622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 421/842] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:10.985284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:11.078379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed9364-b923-4188-a08e-788577e8ecfb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:11.106809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:11.112614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:11.113197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:11.113469 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:11.113735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:11.114371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:11.116678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4841a5-af56-42e6-91aa-3165de60d52a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:11.137588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 422/843] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 02:00:10 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:00:13.278256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc12-4f1a-4805-b827-1d89769e4cd7 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:13.278807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc12-4f1a-4805-b827-1d89769e4cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:13.278899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa4dc12-4f1a-4805-b827-1d89769e4cd7 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:13.280174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 422/844] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:13.285515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:13.285908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:13.286058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:13.305127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:13.306293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:13.371318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 423/845] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:13.642111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4dc12-4f1a-4805-b827-1d89769e4cd7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:13.717627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:13.743380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa4dc12-4f1a-4805-b827-1d89769e4cd7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:13.776777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:13.777455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:13.777777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:13.778056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] PATH_INFO: `/v3/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:13.779435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] RBAC: Authorizing `identity:get_role(role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:13.780798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:13.782107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 423/846] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:13 2022] GET /identity/v3/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:13.788128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:13.883412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2841f6e-bf9f-41d9-b53d-6e1b775f97f5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:13.899245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:13.917217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:13.917834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:13.918111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:13.918386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] PATH_INFO: `/v3/users/8cb91dc5efde491093ce6c88ddedd190` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:13.931490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] RBAC: Authorizing `identity:get_user(user_id=8cb91dc5efde491093ce6c88ddedd190)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:13.932926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:13.934506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 424/847] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:13 2022] GET /identity/v3/users/8cb91dc5efde491093ce6c88ddedd190 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:13.940133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:14.035378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75c251e-726e-4939-bd7c-bd5f7eb8fac0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:14.067717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:14.068333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:14.068611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:14.068885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:14.069969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] RBAC: Authorizing `identity:get_project(project_id=c701720de080402db19dcfb42502520b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:14.071412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba2607-e5fb-4386-9b8f-f52c81b81b5f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:14.072997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 424/848] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:13 2022] GET /identity/v3/projects/c701720de080402db19dcfb42502520b => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:14.078691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:14.173585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075df4f-21d8-4fc4-a78b-a23dd255b753 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:14.205103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:14.205734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:14.206011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:14.206283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b/users/8cb91dc5efde491093ce6c88ddedd190/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:14.208377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] RBAC: Authorizing `identity:create_grant(project_id=c701720de080402db19dcfb42502520b, user_id=8cb91dc5efde491093ce6c88ddedd190, role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:14.210310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55415c-ebe7-47c8-8de6-5aa8ca568806 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:14.223160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 425/849] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:00:14 2022] PUT /identity/v3/projects/c701720de080402db19dcfb42502520b/users/8cb91dc5efde491093ce6c88ddedd190/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:00:16.328822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da916e4-dfba-421e-a3f4-783198c1317c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:16.329344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da916e4-dfba-421e-a3f4-783198c1317c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:16.329344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da916e4-dfba-421e-a3f4-783198c1317c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:16.330728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 425/850] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:16.336166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:16.336562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:16.336715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:16.356001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:16.357163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:16.383252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:16.421645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 426/851] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:16.690084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da916e4-dfba-421e-a3f4-783198c1317c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:16.702716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:16.790637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da916e4-dfba-421e-a3f4-783198c1317c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:16.822860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:16.823468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:16.823747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:16.824019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] PATH_INFO: `/v3/roles/4ba3f1bc788c4045b49a4852f9b61fc7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:16.825372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] RBAC: Authorizing `identity:get_role(role_id=4ba3f1bc788c4045b49a4852f9b61fc7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:16.826752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:16.828053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 426/852] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:16 2022] GET /identity/v3/roles/4ba3f1bc788c4045b49a4852f9b61fc7 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:16.833913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:16.929054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a6806e-26bb-4461-b4f7-3c48be0936db None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:16.961228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:16.961774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:16.962050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:16.962320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] PATH_INFO: `/v3/users/8cb91dc5efde491093ce6c88ddedd190` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:16.963437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] RBAC: Authorizing `identity:get_user(user_id=8cb91dc5efde491093ce6c88ddedd190)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:16.964850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:16.966381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 427/853] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:16 2022] GET /identity/v3/users/8cb91dc5efde491093ce6c88ddedd190 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:16.972336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:17.047945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:17.069669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969e6a79-04e5-43ab-aae7-37578b5ff7a9 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:17.101779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:17.102372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:17.102678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:17.102952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:17.168530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:17.169945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:17.357459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:17.378936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 427/854] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:00:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:00:17.385432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:17.415822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:17.482753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9385905-6ef4-48e9-9648-e7b36ac03e8e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:17.515424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:17.516041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:17.516322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:17.516593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] PATH_INFO: `/v3/projects/c701720de080402db19dcfb42502520b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:17.517685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] RBAC: Authorizing `identity:get_project(project_id=c701720de080402db19dcfb42502520b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:17.519145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab300071-4861-4289-9a36-3af1f0cb309b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:17.520745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 428/855] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:17 2022] GET /identity/v3/projects/c701720de080402db19dcfb42502520b => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:17.527626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:17.620616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce4be2-df0d-4847-a1ec-4d8aea3a337a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:17.653043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:17.653619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:17.653894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:17.654162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:17.654894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] RBAC: Adding query filter params (role.id=4ba3f1bc788c4045b49a4852f9b61fc7, scope.project.id=c701720de080402db19dcfb42502520b, user.id=8cb91dc5efde491093ce6c88ddedd190) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:17.655196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:17.656588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0c04b4-465a-4502-9694-c151e310b402 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:17.662265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 428/856] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:00:17 2022] GET /identity/v3/role_assignments?user.id=8cb91dc5efde491093ce6c88ddedd190&scope.project.id=c701720de080402db19dcfb42502520b&role.id=4ba3f1bc788c4045b49a4852f9b61fc7 => generated 608 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:19.823855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f595f4-b051-4eb3-84d2-9f3d796637e2 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:19.824413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f595f4-b051-4eb3-84d2-9f3d796637e2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:19.824413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f595f4-b051-4eb3-84d2-9f3d796637e2 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:19.825731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 429/857] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:19.830996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:19.831381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:19.831554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:19.850298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:19.851515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:19.885422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:19.915815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 429/858] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:20.184067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f595f4-b051-4eb3-84d2-9f3d796637e2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:20.278155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:20.282201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f595f4-b051-4eb3-84d2-9f3d796637e2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:20.314989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:20.315597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:20.315872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:20.316143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:20.317227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:20.319529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e9945-1aa6-4968-bc5f-37e46e1ad205 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:20.321060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 430/859] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:20.326947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:20.424074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d60e52-a146-4d03-8ef2-77507a7e209c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:20.456564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:20.457095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:20.457371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:20.457645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:20.458273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:20.460260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84afa37-f144-42e6-9e61-bbc3d66cccb7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:20.485761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 430/860] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 02:00:20 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:00:22.629760 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268fb2-40ff-4ef4-a904-d3716b038252 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:22.630311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268fb2-40ff-4ef4-a904-d3716b038252 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:22.630311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14268fb2-40ff-4ef4-a904-d3716b038252 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:22.632169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 431/861] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:22.637391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:22.637807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:22.637936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:22.656818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:22.657987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:22.674373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:22.725567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 431/862] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:22.997052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268fb2-40ff-4ef4-a904-d3716b038252 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:23.094935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14268fb2-40ff-4ef4-a904-d3716b038252 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:23.127394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:23.128008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:23.128289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:23.128564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:23.129661 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:23.131708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c549aa-f0e4-4e5f-a9b3-310ae20b911d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:23.133238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 432/863] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:00:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:23.138939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:23.232433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f4d986-998e-4c80-a4ac-1ffb21149362 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:23.265741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:23.266273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:23.266582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:23.266857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:23.267500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:23.268898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd4e5d6-5d07-40a1-aac3-23f1f07c0a19 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:23.288630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 432/864] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 02:00:23 2022] POST /identity/v3/users => generated 286 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:00:25.446788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede803-cede-49a6-a12d-2023cfe09270 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:25.447310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede803-cede-49a6-a12d-2023cfe09270 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:25.447310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ede803-cede-49a6-a12d-2023cfe09270 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:25.448660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 433/865] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:25.453856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:25.454241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:25.454400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:25.473153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:25.474307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:25.478045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:25.539468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 433/866] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:25.805904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ede803-cede-49a6-a12d-2023cfe09270 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:25.841563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:25.904823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ede803-cede-49a6-a12d-2023cfe09270 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:25.936982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:25.937596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:25.937877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:25.938147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:25.948391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:25.953127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:25.954187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:25.955055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 434/867] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:25.960200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:26.054843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4966762a-70ea-430b-8d09-35c9795b4b96 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:26.086867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:26.087402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:26.087681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:26.087950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:26.088596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:26.088898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:26.090245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:26.098159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 434/868] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:26.104358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:26.195494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:26.201233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f908c1c-c782-4ba3-b1b8-ef58bd0bb1ee None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:26.233265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:26.233876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:26.234156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:26.234431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] PATH_INFO: `/v3/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:26.247248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] RBAC: Authorizing `identity:get_user(user_id=9e0cf0f0ef344bfab6e5ecbd2a0601cd)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:26.248675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:26.250246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 435/869] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:26 2022] GET /identity/v3/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:26.256620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:26.259440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:26.353828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d7f47-0181-4f6b-9d1f-7956e151bce7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:26.385772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:26.386380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:26.386684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:26.386956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] PATH_INFO: `/v3/projects/d42511791493467689435b9ece4ff823` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:26.388047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] RBAC: Authorizing `identity:get_project(project_id=d42511791493467689435b9ece4ff823)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:26.389460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b06054-ad2e-4911-a148-c97296d98c6e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:26.391088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 435/870] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:26 2022] GET /identity/v3/projects/d42511791493467689435b9ece4ff823 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:26.397190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:26.494131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4cce55-5394-4229-9811-81ea3e90057f None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:26.526313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:26.526979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:26.527261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:26.527532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] PATH_INFO: `/v3/projects/d42511791493467689435b9ece4ff823/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:26.529394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] RBAC: Authorizing `identity:create_grant(project_id=d42511791493467689435b9ece4ff823, user_id=9e0cf0f0ef344bfab6e5ecbd2a0601cd, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:26.530831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a93dad-23d9-4efa-93a7-f460f9589380 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:26.543875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 436/871] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:00:26 2022] PUT /identity/v3/projects/d42511791493467689435b9ece4ff823/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:00:28.667311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695429-cb36-473e-992d-38ce050c1b94 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:28.667836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695429-cb36-473e-992d-38ce050c1b94 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:28.667836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa695429-cb36-473e-992d-38ce050c1b94 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:28.669178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 436/872] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:28.674515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:28.674904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:28.675064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:28.694829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:28.696017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:28.705196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:28.762834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 437/873] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:29.028703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695429-cb36-473e-992d-38ce050c1b94 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:29.096396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:29.127292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa695429-cb36-473e-992d-38ce050c1b94 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:29.159387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:29.159995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:29.160275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:29.160548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:29.170230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:29.175004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:29.176058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:29.177083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 437/874] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:29.183131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:29.278249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e00873-876c-4c1d-89fc-b9add855d9d4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:29.310633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:29.311165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:29.311442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:29.311715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:29.312360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:29.312666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:29.314022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:29.322106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 438/875] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:29.327908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:29.425306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85607b18-b1ec-420c-9f70-ef2e97eb9501 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:29.453530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:29.459519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:29.460133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:29.460415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:29.460693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] PATH_INFO: `/v3/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:29.461804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] RBAC: Authorizing `identity:get_user(user_id=9e0cf0f0ef344bfab6e5ecbd2a0601cd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:29.463248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:29.464793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 438/876] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/users/9e0cf0f0ef344bfab6e5ecbd2a0601cd => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:29.470801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:29.480598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:29.568530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10252708-8795-49cf-bec9-48dbdde50903 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:29.600747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:29.601337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:29.601614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:29.601883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:29.665635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:29.667092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:29.674216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:29.762266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 439/877] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:00:29.768119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:29.863234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07203e39-2e48-455c-855d-7af0e19aff66 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:29.894815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:29.895429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:29.895707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:29.895975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] PATH_INFO: `/v3/projects/d42511791493467689435b9ece4ff823` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:29.897708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] RBAC: Authorizing `identity:get_project(project_id=d42511791493467689435b9ece4ff823)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:29.899224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6290dfb1-a641-4d3c-b706-800f79ccf9c5 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:29.900801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 439/878] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/projects/d42511791493467689435b9ece4ff823 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:29.906788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:30.002916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35a3fe-a1c9-439d-8aac-3f31c93b9194 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:30.011267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:30.037634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:30.038208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:30.038513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:30.038786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:30.039490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=d42511791493467689435b9ece4ff823, user.id=9e0cf0f0ef344bfab6e5ecbd2a0601cd) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:30.039796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:30.041200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f6134d-3d54-426e-b00e-789ba3462dc0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:30.046689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 440/879] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:00:29 2022] GET /identity/v3/role_assignments?user.id=9e0cf0f0ef344bfab6e5ecbd2a0601cd&scope.project.id=d42511791493467689435b9ece4ff823&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:32.200579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e49b8-adaa-46ab-9568-071ce1112eb3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:32.201109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e49b8-adaa-46ab-9568-071ce1112eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:32.201109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e49b8-adaa-46ab-9568-071ce1112eb3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:32.202472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 440/880] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:32.207722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73db7f-3a29-4c5e-b493-62ee7cbbb2aa None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:32.208125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73db7f-3a29-4c5e-b493-62ee7cbbb2aa None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:32.208266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73db7f-3a29-4c5e-b493-62ee7cbbb2aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:32.228055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dc73db7f-3a29-4c5e-b493-62ee7cbbb2aa None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:32.229214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dc73db7f-3a29-4c5e-b493-62ee7cbbb2aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:32.297877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 441/881] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Oct 23 02:00:32.579201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e49b8-adaa-46ab-9568-071ce1112eb3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:32.652412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:32.691929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e49b8-adaa-46ab-9568-071ce1112eb3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:32.726234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:32.726858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:32.727135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:32.727405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:32.728102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:32.729448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e6a55e-0a9f-42b4-b4bf-135c0150a299 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:32.746116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 441/882] 10.222.0.23 () {60 vars in 1138 bytes} [Sun Oct 23 02:00:32 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:00:34.929768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4358c44-5911-494f-9bee-6a4d663fb353 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:34.930308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4358c44-5911-494f-9bee-6a4d663fb353 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:34.930308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4358c44-5911-494f-9bee-6a4d663fb353 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:34.931689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 442/883] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:34.937017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:34.937428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:34.937527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:34.955616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:34.956780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:35.021173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 442/884] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:35.287987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4358c44-5911-494f-9bee-6a4d663fb353 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:35.386884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:35.390949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4358c44-5911-494f-9bee-6a4d663fb353 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:35.423480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:35.424096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:35.424377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:35.424651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] PATH_INFO: `/v3/domains/54097ffcc7f9493dae71da89371ed0aa` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:35.434606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=54097ffcc7f9493dae71da89371ed0aa)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:35.436047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9845023d-64ae-4d8f-be61-1a0447afc559 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:35.437615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 443/885] 10.222.0.23 () {56 vars in 1183 bytes} [Sun Oct 23 02:00:35 2022] GET /identity/v3/domains/54097ffcc7f9493dae71da89371ed0aa => generated 267 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:00:35.443953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:35.498144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:35.541652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01d2ea8-1947-407f-9a8f-6600af6b22f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:35.574344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:35.574954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:35.575230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:35.575499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:35.576141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:35.577540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1220679b-6365-456e-a87e-32717f2e01e9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:35.604104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 443/886] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 02:00:35 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:00:37.781474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51362-9247-4dd8-a39d-d5fc6214e619 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:37.782007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51362-9247-4dd8-a39d-d5fc6214e619 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:37.782007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c51362-9247-4dd8-a39d-d5fc6214e619 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:37.783366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 444/887] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:37.788665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:37.789085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:37.789224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:37.807361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:37.808533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:37.872812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 444/888] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:37 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:38.147410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c51362-9247-4dd8-a39d-d5fc6214e619 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:38.247380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c51362-9247-4dd8-a39d-d5fc6214e619 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:38.279919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:38.280530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:38.280809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:38.281080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] PATH_INFO: `/v3/domains/54097ffcc7f9493dae71da89371ed0aa` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:38.282176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] RBAC: Authorizing `identity:get_domain(domain_id=54097ffcc7f9493dae71da89371ed0aa)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:38.283621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe552aa-6565-4241-810d-4c5868c23213 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:38.285164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 445/889] 10.222.0.23 () {56 vars in 1183 bytes} [Sun Oct 23 02:00:38 2022] GET /identity/v3/domains/54097ffcc7f9493dae71da89371ed0aa => generated 267 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:38.291207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:38.329021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:38.387438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43431e49-55d2-4b23-96e4-7254cecf0b03 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:38.420420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:38.420952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:38.421228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:38.421498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:38.422145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:38.423574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5870991b-1c52-4971-bf1b-cbc461d0cc50 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:38.444084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 445/890] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 02:00:38 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:00:40.631574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073c63-09ed-419d-bf63-a5356961cfc7 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:40.632152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073c63-09ed-419d-bf63-a5356961cfc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:40.632152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48073c63-09ed-419d-bf63-a5356961cfc7 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:40.633447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 446/891] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:40.638583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b1266-d569-4497-978b-7a45d408781b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:40.638978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b1266-d569-4497-978b-7a45d408781b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:40.639123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303b1266-d569-4497-978b-7a45d408781b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:40.657865 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-303b1266-d569-4497-978b-7a45d408781b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:40.659055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-303b1266-d569-4497-978b-7a45d408781b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:40.725270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 446/892] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:40.998021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48073c63-09ed-419d-bf63-a5356961cfc7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:41.026235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:41.098403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48073c63-09ed-419d-bf63-a5356961cfc7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:41.133060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:41.133668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:41.133944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:41.134210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:41.143857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:41.149528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:41.150788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:00:41.151956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 447/893] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:41.156625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b1266-d569-4497-978b-7a45d408781b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:41.180781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:41.253199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303b1266-d569-4497-978b-7a45d408781b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:41.285522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:41.286722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:41.287004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:41.287274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:41.287924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:41.288227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:41.289586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:41.297245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 447/894] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:41.303007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:41.389593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:41.402362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b70c94-6117-4ff1-b56f-122412366f2e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:41.435625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:41.436236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:41.436514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:41.436781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] PATH_INFO: `/v3/users/1cba1272826c40b1b6b08c3c421b99c0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:41.450108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] RBAC: Authorizing `identity:get_user(user_id=1cba1272826c40b1b6b08c3c421b99c0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:41.451573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:41.453123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 448/895] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:41 2022] GET /identity/v3/users/1cba1272826c40b1b6b08c3c421b99c0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:41.459758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:41.541063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:41.556336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297d6a4f-9f40-4cf5-9b22-42eaac55f5c2 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:41.589179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:41.589790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:41.590065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:41.590334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] PATH_INFO: `/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:41.591454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] RBAC: Authorizing `identity:get_project(project_id=4d39d25b12754aa6a4cfa35aad2fd7ee)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:41.592854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7655bbe-fe96-4769-81c4-6254f99ae8c8 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:41.594742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 448/896] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:41 2022] GET /identity/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:41.600748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:41.698350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb76454-261f-4b81-aff8-a494801c7459 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:41.731558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:41.732198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:41.732474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:41.732744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] PATH_INFO: `/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee/users/1cba1272826c40b1b6b08c3c421b99c0/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:41.734607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] RBAC: Authorizing `identity:create_grant(project_id=4d39d25b12754aa6a4cfa35aad2fd7ee, user_id=1cba1272826c40b1b6b08c3c421b99c0, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:41.736009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32371e4-d6ff-4b39-99cc-057574f89b23 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:41.748624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 449/897] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:00:41 2022] PUT /identity/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee/users/1cba1272826c40b1b6b08c3c421b99c0/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:00:43.875127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c0be5-f434-410e-84c6-8ab354c13e18 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:43.875664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c0be5-f434-410e-84c6-8ab354c13e18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:43.875664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526c0be5-f434-410e-84c6-8ab354c13e18 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:43.877005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 449/898] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:00:43.883042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:43.883458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:43.883602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:43.896859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:43.902788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:00:43.903988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:00:43.972173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 450/899] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:00:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:00:44.245093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c0be5-f434-410e-84c6-8ab354c13e18 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:44.342836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526c0be5-f434-410e-84c6-8ab354c13e18 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:44.376047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:44.376659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:44.376937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:44.377207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:44.387563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:00:44.392316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:00:44.393405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:00:44.394668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 450/900] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:00:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:00:44.399969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:44.496664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdebb67a-959e-4f57-a13f-d6e9c14e31a8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:44.528681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:44.529273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:44.529569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:44.529840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:44.530519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:44.530824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:44.532178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:44.534265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:44.541133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 451/901] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:00:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:44.547640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:44.562942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:44.644502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6091a2e7-e9d8-4d82-bb32-db8d71426ec1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:44.678014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:44.678653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:44.678935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:44.679209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] PATH_INFO: `/v3/users/1cba1272826c40b1b6b08c3c421b99c0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:44.680337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] RBAC: Authorizing `identity:get_user(user_id=1cba1272826c40b1b6b08c3c421b99c0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:44.681762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:44.683555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 451/902] 10.222.0.23 () {56 vars in 1177 bytes} [Sun Oct 23 02:00:44 2022] GET /identity/v3/users/1cba1272826c40b1b6b08c3c421b99c0 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:44.689380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:44.786109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e5f51-ea8a-4997-880d-b4eb740985e8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:44.820153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:44.820747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:44.821027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:44.821301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:44.883185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:44.886789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:44.888427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:44.981891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 452/903] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:00:44 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:00:44.987797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:45.066674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:45.085065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d9c24e-abc1-4769-833f-5356d8ee5c81 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:45.117293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:45.117904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:45.118184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:45.118476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] PATH_INFO: `/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:45.119572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] RBAC: Authorizing `identity:get_project(project_id=4d39d25b12754aa6a4cfa35aad2fd7ee)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:45.120996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b99fa6-94b5-4094-95f9-b4aa5aacadcf None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:45.122600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 452/904] 10.222.0.23 () {56 vars in 1186 bytes} [Sun Oct 23 02:00:44 2022] GET /identity/v3/projects/4d39d25b12754aa6a4cfa35aad2fd7ee => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:45.128743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:00:45.222225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:45.226868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92809b73-c82a-443b-b930-311d23846d62 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:00:45.260122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:00:45.260704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:00:45.260980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:00:45.261248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:00:45.261952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=4d39d25b12754aa6a4cfa35aad2fd7ee, user.id=1cba1272826c40b1b6b08c3c421b99c0) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:00:45.262257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:00:45.263693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331fb60-070d-4d0c-9834-be4011740ebe None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:00:45.269193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 453/905] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:00:45 2022] GET /identity/v3/role_assignments?user.id=1cba1272826c40b1b6b08c3c421b99c0&scope.project.id=4d39d25b12754aa6a4cfa35aad2fd7ee&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:00:55.113133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:00:55.267223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=19 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:40.513741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8baa6-571d-42ab-b1ba-eac2a9b4074c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:40.514314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8baa6-571d-42ab-b1ba-eac2a9b4074c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:40.514314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd8baa6-571d-42ab-b1ba-eac2a9b4074c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:40.515971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 453/906] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:01:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:01:40.522965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:40.523468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:40.523570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:40.545041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:01:40.546413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:01:40.636652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 454/907] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:01:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:01:40.932534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8baa6-571d-42ab-b1ba-eac2a9b4074c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:41.067294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd8baa6-571d-42ab-b1ba-eac2a9b4074c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:41.106698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:41.107379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:41.107623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:41.107882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:41.119572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:41.121019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:41.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:41.137757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:41.138905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:41.138905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:41.138905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:41.146209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 454/908] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:41.147491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:41.287540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab997-6ad0-44b9-b12f-cf1ae8566df7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:41.320624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:41.326937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:41.327526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:41.327803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:41.328076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:41.328765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:41.329163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:41.331075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a7362d-c4d0-4a33-9ed3-67a9f5811513 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:41.350238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 455/909] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:41.350496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:41.416167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:41.478952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2ec745-e59f-4730-8e24-5f60877d9730 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:41.527174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:41.527174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:41.527174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:41.527174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:41.528037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:41.528037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf2f904-8579-49c9-be16-b02fdd7538bd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:41.555616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 455/910] 10.222.0.23 () {60 vars in 1132 bytes} [Sun Oct 23 02:01:41 2022] POST /identity/v3/users => generated 251 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:01:44.076939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3316-a12c-4f3d-9e3a-6240f0813644 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:44.077473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3316-a12c-4f3d-9e3a-6240f0813644 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:44.077673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdde3316-a12c-4f3d-9e3a-6240f0813644 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:44.080001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 456/911] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:01:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:01:44.085524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:44.085949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:44.086108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:44.107581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:01:44.108785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:01:44.187655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 456/912] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:01:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:01:44.459144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3316-a12c-4f3d-9e3a-6240f0813644 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:44.588983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdde3316-a12c-4f3d-9e3a-6240f0813644 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:44.632452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:44.633345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:44.633451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:44.633643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:44.650474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:01:44.655626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:44.657508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 02:01:44.658934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:44.662253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 457/913] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 02:01:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:44.663555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:44.708039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:44.779084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30461f3f-8030-461b-9a89-19f6ff249df2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:44.826896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:44.827751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:44.828169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:44.829134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:44.829951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] RBAC: Adding query filter params (name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:44.830409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:44.832471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:44.846646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 457/914] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 02:01:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:44.848408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:44.905863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:44.965907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e400a7-80f1-40c4-bb83-e202c934e062 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.013732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.015054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.015490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.015852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.028397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.031199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:45.048894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:45.050357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:45.052001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:45.052001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:45.052001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:45.052001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 458/915] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:45.056699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:45.184022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:45.187347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09598537-b8c1-4042-baa3-1c6dec7b1a03 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.227892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.228595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.228829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.229156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.229800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:45.230176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.231743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:45.247922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 458/916] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:45.255435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:45.368455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870b1a16-d4c8-4f85-8167-4d1715675b02 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.370474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:45.409883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.410705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.411103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.411502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.419691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.421366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:01:45.434356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:45.435565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:45.436626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 459/917] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:45.441531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:45.555948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de4100a-4986-4808-9afa-c154d3107b1b None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.592695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.593302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.593575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.593844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.594722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:45.595071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.596599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:45.619732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 459/918] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Oct 23 02:01:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:45.620907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:45.738900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2d4a19-2b7a-4b94-954b-1239fc647f5b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.782946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.783750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.784137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.784512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.795575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.797542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:45.814148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:45.815394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:45.821106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:45.821106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:45.821106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:45.821106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 460/919] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:45.821474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:45.846404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:45.943478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefdc147-fe08-4b8d-be28-5ad76906f9fa None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:45.989663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:45.990329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:45.990639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:45.990916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:45.991612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:45.991917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:45.993342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:46.013627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 460/920] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:46.015401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:46.034177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:46.127623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee483d75-baac-48dc-9159-032d63d8a5a3 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:46.164580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:46.165209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:46.165513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:46.165763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:46.173189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:46.174678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:01:46.186500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:46.187649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:46.188729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 461/921] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:01:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:46.195455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:46.302147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:46.337630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f45c6-7038-4105-9e98-a9fb136d48b0 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:46.382317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:46.383376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:46.383797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:46.384165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:46.384973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:46.385422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:46.387119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455f9dd-b5c8-49fe-acb8-767b7cf0fa8b None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:46.401853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 461/922] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:01:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:46.408329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:46.505325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:46.538371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab17af4-45cc-42c9-99f0-2fc6fe6611c8 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:46.590297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:46.591007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:46.591287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:46.591843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/c845b9e9ea3f43bcbab59b306d6c5da0/roles/11184e5bad41467fa5d6ccb1a3805e07` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:46.610129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=c845b9e9ea3f43bcbab59b306d6c5da0, role_id=11184e5bad41467fa5d6ccb1a3805e07)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:46.611720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a95664-22f3-470b-9478-19fe7bc307be None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:46.627854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 462/923] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:01:46 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/c845b9e9ea3f43bcbab59b306d6c5da0/roles/11184e5bad41467fa5d6ccb1a3805e07 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:01:49.063450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c166d8-87c9-44b8-9ea5-e2a1ec0bb95d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:49.064037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c166d8-87c9-44b8-9ea5-e2a1ec0bb95d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:49.064037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c166d8-87c9-44b8-9ea5-e2a1ec0bb95d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:49.065247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 462/924] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:01:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:01:49.070951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:49.071442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:49.071636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:49.089559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:01:49.090900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:01:49.158599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 463/925] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:01:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:01:49.444437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c166d8-87c9-44b8-9ea5-e2a1ec0bb95d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:49.561845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c166d8-87c9-44b8-9ea5-e2a1ec0bb95d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:49.601444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:49.602045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:49.602346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:49.602704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:49.610213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:49.614203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:01:49.619668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:49.620792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:49.621731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 463/926] 10.222.0.23 () {56 vars in 1102 bytes} [Sun Oct 23 02:01:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:49.628617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:49.689937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:49.739101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715159e4-d2c5-43f0-a814-ec9c7ba8954b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:49.779830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:49.780434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:49.780714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:49.780991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:49.781782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] RBAC: Adding query filter params (name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:49.782086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:49.783500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:49.791823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 464/927] 10.222.0.23 () {56 vars in 1103 bytes} [Sun Oct 23 02:01:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:49.799153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:49.932145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7152f0e1-cd4a-4df4-a305-0a0681affd3e None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:49.972700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:49.973628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:49.973912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:49.974273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:49.986516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:49.988345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:50.002778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:50.004064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:50.005168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:50.005168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:50.005168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:50.005168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 464/928] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:50.009724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:50.118099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297c6927-1337-4026-afbc-9a7c85a8cc43 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:50.134469 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:50.154010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:50.154647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:50.154924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:50.155196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:50.155838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:50.156138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:50.158058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:50.170348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 465/929] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:50.176672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:50.233405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:50.289816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b11e16d-32cb-4df6-8b74-c0d50ea43e6c None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:50.331815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:50.332598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:50.333022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:50.333437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:50.343199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:50.345284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:01:50.358560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:50.360577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:50.361685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 465/930] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:50.366515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:50.487593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be3234f-8dc5-4f3d-b115-9434781a3f6b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:50.526834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:50.528017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:50.528017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:50.528017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:50.528822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:50.529100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:50.530636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:50.551139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 466/931] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Oct 23 02:01:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:50.556958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:50.664937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a72e7a-407f-4b1a-a871-5daea9c4c265 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:50.676966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:50.705015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:50.705620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:50.705902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:50.706174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:50.775889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:50.777339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:50.887397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 466/932] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:01:50 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:01:50.893520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:50.928600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:50.999410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038259b9-aae8-4a16-829c-249772797e13 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:51.044510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:51.045214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:51.045531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:51.045846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:51.058632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:51.060109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:51.078539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:51.079555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:51.080899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:51.080899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:51.080899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:51.080899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 467/933] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:51.085395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:51.097820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:51.191248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f147f7-2210-4be9-a72a-878c93ae5827 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:51.228215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:51.228901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:51.229241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:51.229607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:51.230425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:51.230840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:51.232398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:51.245029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 467/934] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:51.251476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:51.347050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:51.369234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7728e0d8-1998-462a-a8f7-2eddeefa48f0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:51.415445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:51.416135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:51.416422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:51.416749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:51.424907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:51.426594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:01:51.438606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:51.439705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:51.440931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 468/935] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:01:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:51.446112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:51.525635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:51.558626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b07d946-621a-4775-b8dc-eb212f19202f None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:51.597204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:51.597807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:51.598088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:51.598372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:51.601799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:51.602148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:51.603685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68440876-15cf-4c7b-88ee-2b75af3bfca3 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:51.616890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 468/936] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:01:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:51.622949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:51.731829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ecca5c-f8a1-49b9-ab80-830b273e8724 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:51.769945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:51.770575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:51.770853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:51.771124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:51.771850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] RBAC: Adding query filter params (role.id=11184e5bad41467fa5d6ccb1a3805e07, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=c845b9e9ea3f43bcbab59b306d6c5da0) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:51.772139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:51.773545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ab424a-f9cf-4896-b50a-9389ba544718 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:51.779189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 469/937] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:01:51 2022] GET /identity/v3/role_assignments?user.id=c845b9e9ea3f43bcbab59b306d6c5da0&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=11184e5bad41467fa5d6ccb1a3805e07 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:53.993405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499102-4dbc-4e21-b9b1-04efca4c7784 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:53.993906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499102-4dbc-4e21-b9b1-04efca4c7784 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:53.993906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc499102-4dbc-4e21-b9b1-04efca4c7784 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:53.995244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 469/938] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:01:54.001793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:54.002229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:54.002338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:54.022323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:01:54.023528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:01:54.026290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:54.097999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 470/939] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:01:53 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:01:54.367874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499102-4dbc-4e21-b9b1-04efca4c7784 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:54.480927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc499102-4dbc-4e21-b9b1-04efca4c7784 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:54.514246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:54.521488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:54.522132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:54.522662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:54.523122 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:54.533887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:01:54.538676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:54.539613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:01:54.540605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:54.544849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 470/940] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:01:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:54.545813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:54.649017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c5bad2-8f5a-44ff-af67-46b346161832 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:54.687906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:54.688517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:54.688801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:54.689073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:54.689728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:54.690032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:54.692832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:54.705243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 471/941] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:01:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:54.711133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:54.818683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4c0a6-ed8c-402c-9cc8-62054436e977 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:54.856877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:54.857511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:54.857793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:54.858070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:54.869414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:54.870999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:54.886599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:54.887829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:54.888708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:54.888708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:54.888708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:54.888708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 471/942] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:54.894489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:54.897892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:55.009198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a86f87-57e1-49b6-b108-b39c29e69c21 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.046171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.046915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.047192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.047518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.048221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:55.048531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.049896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.062875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 472/943] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:55.068653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:55.114775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:55.175702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1071a61-d7cb-43dc-9605-5f6d15a54f42 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.211069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.211702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.211980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.212248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] PATH_INFO: `/v3/users/placement` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.222218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.224109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:01:55.238882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:55.240021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:55.241113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 472/944] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:55.246188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:55.313747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:55.353480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df5466-96d8-42aa-b0f7-aafd24262ae6 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.390911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.391473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.391767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.392024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.392749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:55.393050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.394501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.414879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 473/945] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:55.420870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:55.528153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e20321b-7fc0-4968-be54-d9dea9e96ede None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.529724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:55.563112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.563734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.564069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.564295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.574703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.576204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:55.590097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:55.591504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:55.592361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:55.592361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:55.592361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:55.592361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 473/946] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:55.597914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:55.704255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671243ba-9690-4479-a606-1323fcdbd84e None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.727568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:55.742940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.743694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.743975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.744246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.744883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:55.745187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.746573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.759208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 474/947] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:55.765016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:55.879087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b11976-db58-4173-94d9-1dbf6db609e7 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:55.917095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:55.917724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:55.918002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:55.918284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:55.925832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:55.927308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:01:55.939421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:55.940501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:01:55.941393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 474/948] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:55.948747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:56.053902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbd87d-180c-4828-b70f-1d8314d0fe34 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:56.089598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:56.090287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:56.090604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:56.090877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:56.091813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:56.092120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:56.093582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2e3901-1527-4106-b1b6-4aba27d063dd None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:56.106735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 475/949] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:01:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:56.113903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:56.139170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:56.221925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d433e808-d9b3-4f1e-9ca4-d4e246a6d5b1 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:56.260734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:56.261653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:56.261934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:56.262204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] PATH_INFO: `/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/c845b9e9ea3f43bcbab59b306d6c5da0/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:56.264147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] RBAC: Authorizing `identity:create_grant(project_id=52677ee112d04bb4a55f42390446c4aa, user_id=c845b9e9ea3f43bcbab59b306d6c5da0, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:56.265706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c6c92-0965-49e7-ae74-d01593dd5da9 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:56.281064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 475/950] 10.222.0.23 () {58 vars in 1439 bytes} [Sun Oct 23 02:01:56 2022] PUT /identity/v3/projects/52677ee112d04bb4a55f42390446c4aa/users/c845b9e9ea3f43bcbab59b306d6c5da0/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:01:58.501148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06466-ecba-48f3-95e7-4d6900917f37 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:58.501859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06466-ecba-48f3-95e7-4d6900917f37 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:58.501859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae06466-ecba-48f3-95e7-4d6900917f37 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:58.503354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 476/951] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:01:58.508834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:58.509230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:58.509392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:58.529498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:01:58.532181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:01:58.607533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 476/952] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:01:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:01:58.886210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae06466-ecba-48f3-95e7-4d6900917f37 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:58.933363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:58.997437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae06466-ecba-48f3-95e7-4d6900917f37 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.034266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.035076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.035411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.035688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.047557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(role_id) Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Oct 23 02:01:59.053279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:59.054254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:59.055130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 477/953] 10.222.0.23 () {56 vars in 1096 bytes} [Sun Oct 23 02:01:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:59.061018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.064881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:59.173962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47caf760-4be8-4564-b5f5-6fe36c2f3167 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.210778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.211377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.211653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.211919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.212574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:59.212870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:59.214226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:59.223449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 477/954] 10.222.0.23 () {56 vars in 1099 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:59.229495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.338922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:59.342392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee5b02-f166-4c29-8115-08fa293fedc7 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.384639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.385300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.385689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.385842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.397773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:59.399864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:01:59.417374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:59.418673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:01:59.419873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:01:59.419873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:01:59.419873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:59.421994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 478/955] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Oct 23 02:01:59.428179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.498168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:59.537267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914ee4ed-0101-4706-98ac-5e299394979a None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.573428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.574052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.574339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.574642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.575280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:59.575583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:59.576959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:59.589749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 478/956] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:01:59.597089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.709801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c74f38-b772-4cad-8aaf-37f7e4283861 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.751171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.751831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.752194 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.752412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.760960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:59.762771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_user(user_id) Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Oct 23 02:01:59.776859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:01:59.778019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:01:59.784378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 479/957] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:01:59.785186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.898724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21138e36-8d96-445d-a1a8-ee8df5473268 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:01:59.928576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:01:59.938585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:01:59.939125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:01:59.939404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:01:59.939678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:01:59.940420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:01:59.940727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:01:59.942152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:01:59.966897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 479/958] 10.222.0.23 () {56 vars in 1143 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:01:59.968804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:01:59.985762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:00.077708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5805883-9f6a-4d0d-9459-7f25c486f68d None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:00.114868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:00.115497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:00.115777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:00.116078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:00.188018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:00.189531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:00.205086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:00.298851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 480/959] 10.222.0.23 () {58 vars in 1282 bytes} [Sun Oct 23 02:01:59 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Oct 23 02:02:00.304727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:00.413033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0104618-48e5-434f-b81f-0ec50e6b78ce None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:00.450336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:00.450926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:00.451205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:00.451477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:00.461697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:00.463174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Oct 23 02:02:00.477909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:02:00.478985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Oct 23 02:02:00.479910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Oct 23 02:02:00.479910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Oct 23 02:02:00.479910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:02:00.479910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 480/960] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:02:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:02:00.485588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:00.586416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6dc3f7-81c4-482c-9ffc-4cb8fee5ef7a None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:00.599566 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:00.620751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:00.621346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:00.621628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:00.621903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:00.622567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:00.622873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:00.624861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:00.635606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 481/961] 10.222.0.23 () {56 vars in 1109 bytes} [Sun Oct 23 02:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:00.641333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:00.675403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:00.740103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601419-0b24-4ae5-a47c-ad59206a10df None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:00.772503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:00.773149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:00.773426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:00.773698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:00.780040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:00.781504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(project_id) Oct 23 02:02:00.792243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application with Lock( Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:02:00.793292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: ERROR keystone.server.flask.application Oct 23 02:02:00.794566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 481/962] 10.222.0.23 () {56 vars in 1111 bytes} [Sun Oct 23 02:02:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:02:00.799732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:00.894965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e818a12-18f1-4cad-98d7-22128b9fe93b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:00.927195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:00.927782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:00.928060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:00.928329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:00.928996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:00.929296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:00.930749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49dcfe0-e417-4ab9-9bf1-852c2f40137a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:00.942137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 482/963] 10.222.0.23 () {56 vars in 1148 bytes} [Sun Oct 23 02:02:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:00.948088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:01.038463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:01.044252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b61272e-edbb-4c98-8996-27af468a7007 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:01.076645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:01.077277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:01.077656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:01.077929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:01.078661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] RBAC: Adding query filter params (role.id=88ada4935bda41c4bc55eea9219d9fe8, scope.project.id=52677ee112d04bb4a55f42390446c4aa, user.id=c845b9e9ea3f43bcbab59b306d6c5da0) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:01.078969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:01.080367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44336f1c-125c-4095-99c8-5aad80396cf2 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:01.086058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 482/964] 10.222.0.23 () {56 vars in 1374 bytes} [Sun Oct 23 02:02:00 2022] GET /identity/v3/role_assignments?user.id=c845b9e9ea3f43bcbab59b306d6c5da0&scope.project.id=52677ee112d04bb4a55f42390446c4aa&role.id=88ada4935bda41c4bc55eea9219d9fe8 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:03.243493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df3dd-ca87-41c8-b7fb-f8d3da276308 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:03.244017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df3dd-ca87-41c8-b7fb-f8d3da276308 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:03.244017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3df3dd-ca87-41c8-b7fb-f8d3da276308 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:03.245355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 483/965] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:02:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:02:03.250842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:03.251242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:03.251388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:03.269137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:02:03.270316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:02:03.333558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 483/966] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:02:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:02:03.605086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3df3dd-ca87-41c8-b7fb-f8d3da276308 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:03.636387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:03.705540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3df3dd-ca87-41c8-b7fb-f8d3da276308 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:03.739357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:03.739975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:03.740253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:03.740524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:03.741089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:03.742464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:02:03.751736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:02:03.752914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:03.753761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 484/967] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 02:02:03 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:02:03.758811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:03.845974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:03.855070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50008889-28b8-4640-bc4a-9224bbeb702c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:03.887939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:03.888526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:03.888803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:03.889071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:03.890780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:03.891090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:03.892446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809c1b58-d6ce-4bbd-9efb-fe928b8331cd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:03.897025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 484/968] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Oct 23 02:02:03 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:03.902796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:03.993385 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:04.000313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c019a71-8533-465f-865c-2c89cc35c059 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:04.033577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:04.034160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:04.034453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:04.034734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] PATH_INFO: `/v3/services` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:04.035343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:04.035642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:04.036997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58b19de-c1d7-4c68-a6cd-a5df261a23e0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:04.041205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 485/969] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Oct 23 02:02:03 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:06.196160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e017e-97d4-4e22-8336-81e617926455 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:06.196746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e017e-97d4-4e22-8336-81e617926455 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:06.196746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168e017e-97d4-4e22-8336-81e617926455 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:06.198045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 485/970] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:02:06.203926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44d258-4f56-48d6-a0a2-d6b416cd6578 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:06.204362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44d258-4f56-48d6-a0a2-d6b416cd6578 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:06.204461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b44d258-4f56-48d6-a0a2-d6b416cd6578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:06.222323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9b44d258-4f56-48d6-a0a2-d6b416cd6578 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:02:06.223515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9b44d258-4f56-48d6-a0a2-d6b416cd6578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:02:06.288253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 486/971] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:02:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:02:06.562334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e017e-97d4-4e22-8336-81e617926455 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:06.661236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168e017e-97d4-4e22-8336-81e617926455 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:06.693985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:06.694625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:06.694907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:06.695180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:06.695835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:06.697179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227e9c1b-c85f-4680-ac97-d0dc91b4e6dd None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:06.711793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 486/972] 10.222.0.23 () {60 vars in 1141 bytes} [Sun Oct 23 02:02:06 2022] POST /identity/v3/services => generated 246 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:02:08.887633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9f8-44a6-4250-8634-b74c279b1fcb None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:08.888197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9f8-44a6-4250-8634-b74c279b1fcb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:08.888197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c4e9f8-44a6-4250-8634-b74c279b1fcb None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:08.890060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 487/973] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:02:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:02:08.895254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:08.895689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:08.895797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:08.914037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:02:08.915237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:02:08.924188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:08.986884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 487/974] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:02:08 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:02:09.255203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e9f8-44a6-4250-8634-b74c279b1fcb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:09.344295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:09.355121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c4e9f8-44a6-4250-8634-b74c279b1fcb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:09.394325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:09.395084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:09.395417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:09.395766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:09.396450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:09.398150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:02:09.409122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:02:09.410659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:09.411764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 488/975] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 02:02:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:02:09.416550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:09.511843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f86099-d0f8-46b6-bf14-7b17024e6ed3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:09.545123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:09.545747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:09.546022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:09.546292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:09.546950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:09.547273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:09.548664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:09.553552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 488/976] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Oct 23 02:02:09 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:09.559723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:09.657199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd059eed-8751-4fbb-9ab1-a4a2d183511b None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:09.690788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:09.691389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:09.691667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:09.691936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:09.692628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] RBAC: Adding query filter params (interface=public, service_id=be774a8e401546cb9926bc4d6495d161, region_id=RegionOne) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:09.692930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:09.694282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114ae4a1-051e-4a11-af53-9a2537252f1b None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:09.699175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 489/977] 10.222.0.23 () {56 vars in 1251 bytes} [Sun Oct 23 02:02:09 2022] GET /identity/v3/endpoints?service_id=be774a8e401546cb9926bc4d6495d161&interface=public®ion_id=RegionOne => generated 194 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:09.705156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:09.708841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:09.801602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c44c5b71-de49-4e4f-b364-98e4853b9c23 None admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:09.834433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:09.835049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:09.835329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:09.835602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:09.836190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:09.837542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c6fc8-59bc-4685-85dc-ebdbe83fde4e None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:09.845046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 489/978] 10.222.0.23 () {56 vars in 1087 bytes} [Sun Oct 23 02:02:09 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:02:11.979005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31dbee-0c8b-4031-8022-47b10fceb47b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:11.979518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31dbee-0c8b-4031-8022-47b10fceb47b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:11.979518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c31dbee-0c8b-4031-8022-47b10fceb47b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:11.980802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 490/979] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:02:11.986182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:11.986628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:11.986770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:12.004806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:02:12.005965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:02:12.066264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:12.071725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 490/980] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:02:11 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Oct 23 02:02:12.342278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31dbee-0c8b-4031-8022-47b10fceb47b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:12.364749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:12.459054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c31dbee-0c8b-4031-8022-47b10fceb47b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:12.492282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:12.492894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:12.493173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:12.493444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:12.494022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:12.495405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Traceback (most recent call last): Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application rv = self.dispatch_request() Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Oct 23 02:02:12.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._get_service(service_id) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Oct 23 02:02:12.505831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.get_or_create( Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application with Lock( Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self._enter() Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application return self.creator() Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application created_value = creator( Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: ERROR keystone.server.flask.application Oct 23 02:02:12.506700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 491/981] 10.222.0.23 () {56 vars in 1117 bytes} [Sun Oct 23 02:02:12 2022] GET /identity/v3/services/placement => generated 90 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Oct 23 02:02:12.512008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:12.605986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82b8f00-274d-4e5e-b054-f87866394263 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:12.639208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:12.639802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:12.640084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:12.640360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] PATH_INFO: `/v3/services` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:12.640980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] RBAC: Adding query filter params (name=placement) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:02:12.641284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:12.642672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1c2e4-0236-4b4b-bbee-e7385286bf2c None admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:12.647618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 491/982] 10.222.0.23 () {56 vars in 1116 bytes} [Sun Oct 23 02:02:12 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:02:12.653198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:02:12.736092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:12.754579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7f5467-e886-404b-92f3-58b34f26bbd0 None admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:02:12.788087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:02:12.788674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:02:12.788949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:02:12.789218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:02:12.789862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:02:12.791232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003d1a59-01ce-49c5-a4c7-da4e35a0700a None admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:02:12.811777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 492/983] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:02:12 2022] POST /identity/v3/endpoints => generated 331 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:02:22.644652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=69 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:02:22.807115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:16.886815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbabf-ca86-42ac-83bf-639120342e94 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:16.887283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbabf-ca86-42ac-83bf-639120342e94 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:16.887530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbbabf-ca86-42ac-83bf-639120342e94 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:16.892604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 492/984] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:16 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:16.894763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d19-9690-4f00-b0da-8222b4435aeb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:16.895168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d19-9690-4f00-b0da-8222b4435aeb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:16.895796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c640d19-9690-4f00-b0da-8222b4435aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:17.053428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9c640d19-9690-4f00-b0da-8222b4435aeb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:17.054670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9c640d19-9690-4f00-b0da-8222b4435aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:17.194776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 493/985] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:03:17.491060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:17.491355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:17.491639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:17.493734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 493/986] 10.222.0.23 () {54 vars in 964 bytes} [Sun Oct 23 02:03:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:17.499895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e26-938e-4520-87cb-3b3dfd89b218 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:17.500189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e26-938e-4520-87cb-3b3dfd89b218 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:17.500470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7181e26-938e-4520-87cb-3b3dfd89b218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:17.591440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d7181e26-938e-4520-87cb-3b3dfd89b218 None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:17.592684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d7181e26-938e-4520-87cb-3b3dfd89b218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:17.616059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:17.681654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 494/987] 10.222.0.23 () {58 vars in 1061 bytes} [Sun Oct 23 02:03:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:17.690120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:17.691929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:17.818337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e17759-1d7d-413b-a784-15e05579ccc4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:17.903022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:17.903702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:17.903993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:17.904149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:17.905911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:17.964715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:18.023188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:18.025098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2ecac8-9a7b-4f9c-8328-a7f4274ee5c7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:18.044354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 494/988] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:20.569821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:20.570349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:20.570349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:20.572420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 495/989] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:20.577712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674523ea-d040-47f6-9961-cbd0225c59a5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:20.578129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674523ea-d040-47f6-9961-cbd0225c59a5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:20.578270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674523ea-d040-47f6-9961-cbd0225c59a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:20.659284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-674523ea-d040-47f6-9961-cbd0225c59a5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:20.660424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-674523ea-d040-47f6-9961-cbd0225c59a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:20.761265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 495/990] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:21.049942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:21.051514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:21.153525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36530422-a1f5-4e77-be65-34d85c21734a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:21.223374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:21.225768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:21.227158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:21.227584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:21.228052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:21.229866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:21.336956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:21.338479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0617bbf6-c8a7-4cce-af23-935f36c1b749 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:21.356668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 496/991] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:23.803007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:23.803529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:23.803529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:23.804897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 496/992] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:23.810750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671426-1c38-4f28-b1d5-959f721d1e55 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:23.811158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671426-1c38-4f28-b1d5-959f721d1e55 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:23.811302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1671426-1c38-4f28-b1d5-959f721d1e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:23.876180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c1671426-1c38-4f28-b1d5-959f721d1e55 None None] MFA Rules not processed for user `758b98dda3db4b77bfb2d5ca9ed32252`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:23.877371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1671426-1c38-4f28-b1d5-959f721d1e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:23.927226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:23.971693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 497/993] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:23 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:24.256582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:24.258736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:24.269158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:24.361057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d86ae9-4698-44cc-91fb-29aba797e85b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:24.435014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:24.435617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:24.435894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:24.436162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:24.437729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:24.498677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:24.544029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:24.545397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae0288-2eb5-4b22-a165-50531da219ef service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:24.563307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 497/994] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:24 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:27.281019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837191-c50e-4008-9f23-137b2356e18b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:27.281556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837191-c50e-4008-9f23-137b2356e18b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:27.281556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3837191-c50e-4008-9f23-137b2356e18b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:27.282980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 498/995] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:27.288182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36910b54-67bf-4d48-b2cb-611b3ffe9444 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:27.289152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36910b54-67bf-4d48-b2cb-611b3ffe9444 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:27.289377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36910b54-67bf-4d48-b2cb-611b3ffe9444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:27.353840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-36910b54-67bf-4d48-b2cb-611b3ffe9444 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:27.355049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36910b54-67bf-4d48-b2cb-611b3ffe9444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:27.453268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 498/996] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:27 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:27.751934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:27.752251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:27.752502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:27.753839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 499/997] 10.222.0.23 () {54 vars in 964 bytes} [Sun Oct 23 02:03:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:27.759567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a23a-11d5-4ef6-8d72-c43381e66bea None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:27.759861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a23a-11d5-4ef6-8d72-c43381e66bea None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:27.760140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89a23a-11d5-4ef6-8d72-c43381e66bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:27.781955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:27.837004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9b89a23a-11d5-4ef6-8d72-c43381e66bea None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:27.838205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b89a23a-11d5-4ef6-8d72-c43381e66bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:27.912648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 499/998] 10.222.0.23 () {58 vars in 1061 bytes} [Sun Oct 23 02:03:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:27.922460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:27.923969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:28.025100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ce43ab-11f9-4753-898f-6f74117241d5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:28.096540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:28.097166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:28.097445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:28.097722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:28.099337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:28.106256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:28.207178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:28.208603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f90048-2255-4198-a289-583261ee7a93 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:28.227116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 500/999] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:27 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:31.534695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:31.536353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:31.536786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:31.539024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 500/1000] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:31.544317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e4ab-7470-4b3e-8e84-81216df83cbb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:31.544831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e4ab-7470-4b3e-8e84-81216df83cbb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:31.544997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af33e4ab-7470-4b3e-8e84-81216df83cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:31.618466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-af33e4ab-7470-4b3e-8e84-81216df83cbb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:31.619712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-af33e4ab-7470-4b3e-8e84-81216df83cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:31.688920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:31.730491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 501/1001] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:32.013311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:32.014909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:32.119578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dcddff-45fa-42c3-afc4-cad8c37ec5b4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:32.133439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:32.195453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:32.196048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:32.196324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:32.196594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:32.198155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:32.306553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:32.307904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed986e43-1b3d-4a2c-bc4a-2ec4d51089df service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:32.326422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 501/1002] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:34.545303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:34.545854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:34.545854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:34.547221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 502/1003] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:34.552398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6853-98da-416b-8283-2b87d3b851ad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:34.552836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6853-98da-416b-8283-2b87d3b851ad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:34.552944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa6853-98da-416b-8283-2b87d3b851ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:34.601190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:34.620037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dcaa6853-98da-416b-8283-2b87d3b851ad None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:34.621229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dcaa6853-98da-416b-8283-2b87d3b851ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:34.728551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 502/1004] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:35.011240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:35.012780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:35.117606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5e3ef5-b1f6-4bc9-9793-a72da138afd0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:35.191905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:35.192510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:35.192787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:35.193054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:35.195373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:35.212709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:35.305115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:35.306463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287950b-8879-468c-b156-710b7217ea43 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:35.325053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 503/1005] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:37.658017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:37.658595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:37.658595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:37.659950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 503/1006] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:37.666262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb876-78bd-4c13-b080-8614f208719f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:37.666744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb876-78bd-4c13-b080-8614f208719f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:37.666846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37cb876-78bd-4c13-b080-8614f208719f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:37.735606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c37cb876-78bd-4c13-b080-8614f208719f None None] MFA Rules not processed for user `758b98dda3db4b77bfb2d5ca9ed32252`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:37.736959 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c37cb876-78bd-4c13-b080-8614f208719f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:37.800429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:37.832178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 504/1007] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:37 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:03:38.118913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:38.120462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:38.227211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a7fe9d-a66d-4363-aa7b-d59b9d81b554 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:38.246728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:38.301988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:38.302635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:38.302919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:38.303883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:38.305431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:38.417522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:38.418910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1462d23f-6bcd-471f-9d5b-e286e7ed7e5a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:38.437301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 504/1008] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:40.848697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:40.849240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:40.849240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:40.851527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 505/1009] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:40.856720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab1de-7748-4765-ac70-f02248d93b0b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:40.857164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab1de-7748-4765-ac70-f02248d93b0b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:40.857270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14ab1de-7748-4765-ac70-f02248d93b0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:40.921302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:40.924143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d14ab1de-7748-4765-ac70-f02248d93b0b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:40.925932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d14ab1de-7748-4765-ac70-f02248d93b0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:41.022485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 505/1010] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:41.309893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:41.311482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:41.414167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3300b16d-cbe7-4982-aed4-de3e4fec46ee None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:41.486637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:41.487234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:41.487515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:41.487792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:41.489373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:41.516756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:41.599501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:41.600839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bfd619f-b709-49e0-9d78-31ae98220f5d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:41.619619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 506/1011] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:43.821752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:43.822275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:43.822389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:43.823754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 506/1012] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:43.828892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7be2-ce2a-4b62-8bf9-364209feb7a0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:43.829317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7be2-ce2a-4b62-8bf9-364209feb7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:43.829467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0e7be2-ce2a-4b62-8bf9-364209feb7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:43.895909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-af0e7be2-ce2a-4b62-8bf9-364209feb7a0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:43.897615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-af0e7be2-ce2a-4b62-8bf9-364209feb7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:43.973874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:44.006619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 507/1013] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:44.285414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:44.286953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:44.389736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297bf8f2-8026-48ac-9457-7f201a722387 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:44.417424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:44.462295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:44.462916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:44.463193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:44.463460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:44.465009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:44.570393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:44.571762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d8bc00-de5f-48c7-801c-1cf7b52803c0 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:44.589673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 507/1014] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:47.168487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:47.168946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:47.169038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:47.170373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 508/1015] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:47.175470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499566c3-bb8d-4cf0-888b-74c2f9667f0d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:47.175883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499566c3-bb8d-4cf0-888b-74c2f9667f0d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:47.176013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499566c3-bb8d-4cf0-888b-74c2f9667f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:47.243992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-499566c3-bb8d-4cf0-888b-74c2f9667f0d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:47.245180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-499566c3-bb8d-4cf0-888b-74c2f9667f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:47.253649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:47.341564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 508/1016] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:47 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:03:47.625082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:47.626636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:47.728960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538e2d77-963a-46cf-b47c-3f3136287f1b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:47.801263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:47.801857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:47.802139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:47.802473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:47.804055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:47.839627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:47.912623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:47.913963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7ef206-226f-4d5f-b6cf-88d500df8351 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:47.931701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 509/1017] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:47 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:52.046062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:52.046589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:52.046589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:52.048180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 509/1018] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:52.053976 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4317-802d-48d3-99bc-b1fa8c3c8b73 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:52.054407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4317-802d-48d3-99bc-b1fa8c3c8b73 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:52.054603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cf4317-802d-48d3-99bc-b1fa8c3c8b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:52.119223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-73cf4317-802d-48d3-99bc-b1fa8c3c8b73 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:52.120418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73cf4317-802d-48d3-99bc-b1fa8c3c8b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:52.205052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:52.217233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 510/1019] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:52.505835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:52.507414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:52.613987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d27cc2f-d6de-4653-baef-b277c7ea469a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:52.654907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:52.694373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:52.695014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:52.695290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:52.695579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:52.697165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:52.806935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:52.808271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e213ae17-f100-4a24-8eec-11ddddd0c63f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:52.826233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 510/1020] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:03:55.575758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:55.576256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:55.576256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:55.577672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 511/1021] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:03:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:03:55.582666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17214ac9-e61b-4c16-b816-4214cc7c47ab None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:55.583630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17214ac9-e61b-4c16-b816-4214cc7c47ab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:55.583793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17214ac9-e61b-4c16-b816-4214cc7c47ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:55.649471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-17214ac9-e61b-4c16-b816-4214cc7c47ab None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:03:55.650694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-17214ac9-e61b-4c16-b816-4214cc7c47ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:55.668764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:55.748831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 511/1022] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:03:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:03:56.038387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:03:56.039978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:56.148491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b07774-9ede-4949-ab50-1012c63cd785 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:03:56.227675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:03:56.228277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:03:56.228560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:03:56.228836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:03:56.230416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:03:56.272929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:03:56.343927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:03:56.345278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc18ec6d-89ef-430a-ad44-fcdbe0f965a1 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:03:56.364617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 512/1023] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:03:56 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:00.595060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:00.595651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:00.595651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:00.596867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 512/1024] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:00.603325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca08503-cb85-4526-a9b5-78e486057fa9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:00.603790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca08503-cb85-4526-a9b5-78e486057fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:00.603904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca08503-cb85-4526-a9b5-78e486057fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:00.672938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bca08503-cb85-4526-a9b5-78e486057fa9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:00.674155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bca08503-cb85-4526-a9b5-78e486057fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:00.790540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:00.794486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 513/1025] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:04:01.080616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:01.082142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:01.184797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee226599-a207-4672-99db-7b644f0dc301 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:01.234727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:01.260741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:01.261359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:01.261629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:01.261929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:01.263584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:01.370318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:01.371767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923e811-78a2-4f71-96fe-63e62a486d75 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:01.390205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 513/1026] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:06.366547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ec81c-52d0-44c9-b680-aa3f0bb5fc83 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:06.366547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ec81c-52d0-44c9-b680-aa3f0bb5fc83 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:06.366547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ec81c-52d0-44c9-b680-aa3f0bb5fc83 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:06.366547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 514/1027] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:04:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:06.380008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4d58e-7445-4bb5-a1dd-5888ca7a06e6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:06.380481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4d58e-7445-4bb5-a1dd-5888ca7a06e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:06.380571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d4d58e-7445-4bb5-a1dd-5888ca7a06e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:06.503039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-52d4d58e-7445-4bb5-a1dd-5888ca7a06e6 None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:06.504902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-52d4d58e-7445-4bb5-a1dd-5888ca7a06e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:06.525099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:06.624972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 514/1028] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:04:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:10.748071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4bba-dfb1-48aa-aa7e-c3d04bee1dcc None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:10.748659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4bba-dfb1-48aa-aa7e-c3d04bee1dcc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:10.748659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d4bba-dfb1-48aa-aa7e-c3d04bee1dcc None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:10.750120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 515/1029] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:04:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:10.756086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988bcd-fd28-4909-a485-5957c59933ac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:10.756587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988bcd-fd28-4909-a485-5957c59933ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:10.756677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f988bcd-fd28-4909-a485-5957c59933ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:10.838084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0f988bcd-fd28-4909-a485-5957c59933ac None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:10.839351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0f988bcd-fd28-4909-a485-5957c59933ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:10.915115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:10.954057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 515/1030] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:04:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:12.339540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807749-8aa7-4c31-8623-0a8b2ca41562 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:12.340090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807749-8aa7-4c31-8623-0a8b2ca41562 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:12.340297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79807749-8aa7-4c31-8623-0a8b2ca41562 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:12.341709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 516/1031] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:04:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:12.346824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36041a35-7aef-4cdb-9911-d8e7751b64c1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:12.347524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36041a35-7aef-4cdb-9911-d8e7751b64c1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:12.347977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36041a35-7aef-4cdb-9911-d8e7751b64c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:12.427428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-36041a35-7aef-4cdb-9911-d8e7751b64c1 None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:12.428636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36041a35-7aef-4cdb-9911-d8e7751b64c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:12.539682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 516/1032] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:04:12 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:16.074287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d1cc-1aa0-458b-91fa-3492916f3c9f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:16.074891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d1cc-1aa0-458b-91fa-3492916f3c9f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:16.075131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382d1cc-1aa0-458b-91fa-3492916f3c9f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:16.076686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 517/1033] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:16.083301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e089-71dd-4302-a73c-ce01988c2594 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:16.084624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e089-71dd-4302-a73c-ce01988c2594 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:16.085110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d9e089-71dd-4302-a73c-ce01988c2594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:16.131206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:16.165681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-66d9e089-71dd-4302-a73c-ce01988c2594 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:16.167050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66d9e089-71dd-4302-a73c-ce01988c2594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:16.271797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 517/1034] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:16.578769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:16.579405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:16.579858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:16.582579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 518/1035] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:04:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:16.590735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa03db-d2a9-49d5-a25a-9e00602f9176 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:16.591463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa03db-d2a9-49d5-a25a-9e00602f9176 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:16.591979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa03db-d2a9-49d5-a25a-9e00602f9176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:16.677877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aaaa03db-d2a9-49d5-a25a-9e00602f9176 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:16.682075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aaaa03db-d2a9-49d5-a25a-9e00602f9176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:16.743617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:16.822327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 518/1036] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:04:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:16.830402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:16.832137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:16.984620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9782a140-0fd6-4d06-b46a-900ee4e22bb4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:17.063143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:17.120110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:17.121111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:17.121504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:17.121801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:17.123877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:17.229285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:17.231293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff8c608-4093-4fbd-9dce-9cc4b10b100e service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:17.255511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 519/1037] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:19.666722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:19.667263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:19.667263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:19.668623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 519/1038] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:19.674076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be578de-91ff-4e15-92cc-a55313cb6d43 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:19.675321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be578de-91ff-4e15-92cc-a55313cb6d43 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:19.675321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be578de-91ff-4e15-92cc-a55313cb6d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:19.747232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1be578de-91ff-4e15-92cc-a55313cb6d43 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:19.748744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1be578de-91ff-4e15-92cc-a55313cb6d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:19.788570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:19.852542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 520/1039] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:20.151744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:20.153851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:20.341112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c56d774-8929-4756-b168-3b944e908de1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:20.354352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:20.481734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:20.482345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:20.482663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:20.482939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:20.484533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:20.578325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:20.579997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300734ea-3759-4a47-a40c-813e0085a863 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:20.599487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 520/1040] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:23.138754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003acf0-32c1-4268-b414-2290c2d16b3e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:23.139293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003acf0-32c1-4268-b414-2290c2d16b3e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:23.139590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8003acf0-32c1-4268-b414-2290c2d16b3e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:23.141195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 521/1041] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:23.147759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124a60-c455-431a-8ec9-4d0bb61fa9fc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:23.148212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124a60-c455-431a-8ec9-4d0bb61fa9fc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:23.148293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe124a60-c455-431a-8ec9-4d0bb61fa9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:23.165118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:23.222642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fe124a60-c455-431a-8ec9-4d0bb61fa9fc None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:23.223841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fe124a60-c455-431a-8ec9-4d0bb61fa9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:23.328553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 521/1042] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:23.630121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:23.630426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:23.630757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:23.632283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 522/1043] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:04:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:23.637602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf5a55-0bc8-486e-b5ae-83cf5187b324 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:23.638159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf5a55-0bc8-486e-b5ae-83cf5187b324 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:23.638634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf5a55-0bc8-486e-b5ae-83cf5187b324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:23.726650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-dddf5a55-0bc8-486e-b5ae-83cf5187b324 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:23.728035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-dddf5a55-0bc8-486e-b5ae-83cf5187b324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:23.764920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:23.853369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 522/1044] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:04:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:23.861376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:23.862937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:24.023354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ec2140-b4a6-4257-a14d-83acb2a1583f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:24.045688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:24.153776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:24.154700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:24.154700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:24.154996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:24.156707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:24.258177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:24.259796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfae37d-cac2-414f-a59d-35899c914be4 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:24.272253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316367-12d8-47b9-8356-8422442b4e69 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:24.272773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316367-12d8-47b9-8356-8422442b4e69 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:24.272856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91316367-12d8-47b9-8356-8422442b4e69 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:24.274355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 523/1045] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:04:24.279249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 523/1046] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:24.279385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becdf0-c90d-4cc6-92a9-ac5e2b58c016 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:24.279493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becdf0-c90d-4cc6-92a9-ac5e2b58c016 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:24.279712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77becdf0-c90d-4cc6-92a9-ac5e2b58c016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:24.358976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-77becdf0-c90d-4cc6-92a9-ac5e2b58c016 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:24.360325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-77becdf0-c90d-4cc6-92a9-ac5e2b58c016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:24.482802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:24.493037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 524/1047] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:04:24.831175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:24.831175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:24.831608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:24.833831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 524/1048] 10.222.0.23 () {54 vars in 963 bytes} [Sun Oct 23 02:04:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:24.838561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108268d-6d36-4ec9-bbea-2f0fd0357546 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:24.838915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108268d-6d36-4ec9-bbea-2f0fd0357546 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:24.839257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1108268d-6d36-4ec9-bbea-2f0fd0357546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:24.922652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1108268d-6d36-4ec9-bbea-2f0fd0357546 None None] MFA Rules not processed for user `a323cd646bae44b5a6f63714b5cdf70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:24.924787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1108268d-6d36-4ec9-bbea-2f0fd0357546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:25.009871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 525/1049] 10.222.0.23 () {58 vars in 1060 bytes} [Sun Oct 23 02:04:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:25.017908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:25.019557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:25.050960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:25.131567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5cea85-148f-4f95-8c9a-ab8fa2eefff9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:25.219623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:25.221573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:25.222022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:25.222553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:25.224790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:25.339025 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:25.390811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:25.393582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64510049-0249-40bd-b534-ef8012fc3daf service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:25.421556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 525/1050] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:26.546576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:26.547133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:26.547133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:26.548550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 526/1051] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:26.554069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961979e3-ab00-4d8b-9c6b-5efa9a2f7f03 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:26.554512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961979e3-ab00-4d8b-9c6b-5efa9a2f7f03 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:26.554814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961979e3-ab00-4d8b-9c6b-5efa9a2f7f03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:26.621306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-961979e3-ab00-4d8b-9c6b-5efa9a2f7f03 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:26.622663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-961979e3-ab00-4d8b-9c6b-5efa9a2f7f03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:26.728255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 526/1052] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:27.027653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:27.029173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:27.121333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:27.187502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20b4ac-fa40-4eed-878c-63b9bb8d8943 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:27.305801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:27.306685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:27.306685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:27.306957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:27.308657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:27.360644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:27.410808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:27.412414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c233735-9153-4386-b2a1-eb88848563c7 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:27.432949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 527/1053] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:29.701159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:29.701775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:29.701775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:29.703443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 527/1054] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:29.710219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45b505-12b9-4a96-a3ff-f6de9ecfd1fb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:29.710827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45b505-12b9-4a96-a3ff-f6de9ecfd1fb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:29.710827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45b505-12b9-4a96-a3ff-f6de9ecfd1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:29.782116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6c45b505-12b9-4a96-a3ff-f6de9ecfd1fb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:29.783507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6c45b505-12b9-4a96-a3ff-f6de9ecfd1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:29.884060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 528/1055] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:30.175784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:30.177477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:30.217104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:30.332549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb7a0f-1509-44f8-872c-32c0c667a59e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:30.450650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:30.457492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:30.458225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:30.458657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:30.459041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:30.460758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:30.557261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:30.558910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfa2457-faf3-47cf-b4f7-c55b879d756d service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:30.579624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 528/1056] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:32.813334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:32.813811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:32.814651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:32.816333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 529/1057] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:32.821631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cccfdd-d4f0-432e-81a8-ae95f1a00143 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:32.821631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cccfdd-d4f0-432e-81a8-ae95f1a00143 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:32.822208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cccfdd-d4f0-432e-81a8-ae95f1a00143 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:32.908658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-83cccfdd-d4f0-432e-81a8-ae95f1a00143 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:32.910822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83cccfdd-d4f0-432e-81a8-ae95f1a00143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:33.021141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:33.058273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 529/1058] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:33.353882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:33.355960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:33.391575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:33.514273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efc59f7-eec6-4482-961a-39121568c30b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:33.621954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:33.639148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:33.639778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:33.640050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:33.640328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:33.642029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:33.739970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:33.741639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc0030f-dbb1-450d-8d08-0fb3a46c7c59 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:33.760714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 530/1059] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:36.251920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:36.252617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:36.252617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:36.254337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 530/1060] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:36.260029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4a31f-d739-4f20-9f7b-4390a9117b7f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:36.260587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4a31f-d739-4f20-9f7b-4390a9117b7f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:36.260719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d4a31f-d739-4f20-9f7b-4390a9117b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:36.326825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-66d4a31f-d739-4f20-9f7b-4390a9117b7f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:36.328037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66d4a31f-d739-4f20-9f7b-4390a9117b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:36.397937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:36.438999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 531/1061] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:36.742930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:36.744627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:36.898913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9c089e-9a84-4d5a-aedf-cd64bdca367f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:36.959457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:37.023369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:37.024001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:37.024295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:37.024582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:37.026244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:37.126683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:37.128838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f35f8ca-fee1-41c7-a761-6785fca193cd service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:37.152677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 531/1062] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:39.596943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:39.597540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:39.597540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:39.598975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 532/1063] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:39.604359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1600-c8ad-4f15-a32a-fe85e1cf789f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:39.604815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1600-c8ad-4f15-a32a-fe85e1cf789f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:39.604983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fc1600-c8ad-4f15-a32a-fe85e1cf789f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:39.669753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-40fc1600-c8ad-4f15-a32a-fe85e1cf789f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:39.670990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-40fc1600-c8ad-4f15-a32a-fe85e1cf789f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:39.679901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:39.772922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 532/1064] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:40.064943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:40.066463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:40.213034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0d2f4-9ec1-4f9c-a2a3-c4aa856c183f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:40.260814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:40.332043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:40.332897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:40.333174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:40.333443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:40.335031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:40.425127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:40.426747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110594c2-5b64-4348-8492-3fcf825a6952 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:40.444852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 533/1065] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:40.794701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353ed3-58e1-4f40-adea-206c7aa9d95b None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:40.795256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353ed3-58e1-4f40-adea-206c7aa9d95b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:40.795378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e353ed3-58e1-4f40-adea-206c7aa9d95b None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:40.796655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 533/1066] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:40.803127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c28-4f3c-4c1d-b3ba-5033a1bca1b9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:40.803609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c28-4f3c-4c1d-b3ba-5033a1bca1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:40.803771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48158c28-4f3c-4c1d-b3ba-5033a1bca1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:40.869435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:40.872374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-48158c28-4f3c-4c1d-b3ba-5033a1bca1b9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:40.874236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48158c28-4f3c-4c1d-b3ba-5033a1bca1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:40.973892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 534/1067] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:41.276419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:41.276813 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:41.277157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:41.278592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 534/1068] 10.222.0.23 () {54 vars in 962 bytes} [Sun Oct 23 02:04:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:41.284580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d7b43-badc-4e8d-bfb8-f6735092cde5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:41.284937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d7b43-badc-4e8d-bfb8-f6735092cde5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:41.285280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2d7b43-badc-4e8d-bfb8-f6735092cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:41.364152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d2d7b43-badc-4e8d-bfb8-f6735092cde5 None None] MFA Rules not processed for user `19022fccab844d1dac60f5eddf67f323`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:41.365395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d2d7b43-badc-4e8d-bfb8-f6735092cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:41.439359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:41.446059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 535/1069] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Oct 23 02:04:41 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:41.455157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:41.456918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:41.559564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e71ba7-1ad5-4061-9242-cc6b17e0bd77 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:41.598022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:41.636475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:41.637089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:41.637369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:41.637641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:41.639308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:41.754901 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:41.756384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa3000b-e554-465f-bc59-805307300641 service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:41.774350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 535/1070] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:42.520147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29046-2fec-4b0f-a198-68e4db6e2737 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:42.520616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29046-2fec-4b0f-a198-68e4db6e2737 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:42.520859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa29046-2fec-4b0f-a198-68e4db6e2737 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:42.597527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aaa29046-2fec-4b0f-a198-68e4db6e2737 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:42.598742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aaa29046-2fec-4b0f-a198-68e4db6e2737 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:42.661225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f528259-65d0-4637-83bb-30070a89221e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:42.661818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f528259-65d0-4637-83bb-30070a89221e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:42.661818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f528259-65d0-4637-83bb-30070a89221e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:42.663148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 536/1071] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:42.668324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:42.668601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:42.668874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:42.685769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 536/1072] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:04:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:42.729912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f6cc-1081-4ed2-822c-53b483f34cb4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:42.730125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f6cc-1081-4ed2-822c-53b483f34cb4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:42.730457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590f6cc-1081-4ed2-822c-53b483f34cb4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:42.732162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 537/1073] 10.222.0.23 () {54 vars in 943 bytes} [Sun Oct 23 02:04:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:04:42.740091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:42.740382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:42.740667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:42.745425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:42.746666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:42.764319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:42.819334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] MFA Rules not processed for user `e03f5a82065a41508004af126326e8ca`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:42.820554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:42.846579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 537/1074] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:04:42.852422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:42.932202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 538/1075] 10.222.0.23 () {58 vars in 1040 bytes} [Sun Oct 23 02:04:42 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:04:42.941797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:42.943082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:43.048601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ec746d-50c4-4e63-a71c-dbae47c34af9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:43.096779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:43.138652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:43.139850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:43.153740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:43.154351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:43.154662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:43.154936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:43.157151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:43.241693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0faa3b-43d8-4776-a97d-04bdf9342fbf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:43.243764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:43.245360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c065c8-20e9-406d-8281-43763b3875e0 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:43.253866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 538/1076] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:04:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:04:43.294521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:43.368766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:43.369379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:43.369655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:43.369925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:43.371775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:43.486713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:43.488300 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fcb5a3-93a5-4454-93dd-deefedae3952 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:43.514755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 539/1077] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:04:45.776506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:45.777067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:45.777067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:45.778642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 539/1078] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:45.787140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50f758-8cc9-481a-bde1-34e4b1598096 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:45.787655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50f758-8cc9-481a-bde1-34e4b1598096 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:45.787771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50f758-8cc9-481a-bde1-34e4b1598096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:45.862853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9b50f758-8cc9-481a-bde1-34e4b1598096 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:45.864397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9b50f758-8cc9-481a-bde1-34e4b1598096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:45.867503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:45.976001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 540/1079] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:46.273047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:46.274692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:46.366036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:46.478095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251fcff1-5f95-4105-bf8e-083806d8a90c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:46.608435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:46.609889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:46.610370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:46.610675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:46.612265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:46.659554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:46.716108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:46.717752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcfc45-b0ab-4dfc-95ce-47aa82af6f94 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:46.738951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 540/1080] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:48.989361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:48.989893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:48.989893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:48.991200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 541/1081] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:48.996488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e111f-9945-4d9a-8fbe-8c83775fa17b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:48.996918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e111f-9945-4d9a-8fbe-8c83775fa17b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:48.997033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29e111f-9945-4d9a-8fbe-8c83775fa17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:49.071092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c29e111f-9945-4d9a-8fbe-8c83775fa17b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:49.072314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c29e111f-9945-4d9a-8fbe-8c83775fa17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:49.180623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 541/1082] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:49.474569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:49.476088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:49.628444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:49.642175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7073d42-5b50-4cea-919c-62368dc0d54e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:49.784315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:49.784843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:49.785121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:49.785391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:49.787120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:49.895131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:49.896741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4056b4f5-ca47-467d-8dff-042ec76e2963 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:49.906395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:49.917218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 542/1083] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:04:52.209845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:52.210411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:52.210411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:52.211749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 542/1084] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:04:52.217158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b4fa-a186-41e4-b0de-36bfc0585afb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:52.217628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b4fa-a186-41e4-b0de-36bfc0585afb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:52.217838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a0b4fa-a186-41e4-b0de-36bfc0585afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:52.296178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-81a0b4fa-a186-41e4-b0de-36bfc0585afb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:04:52.297508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-81a0b4fa-a186-41e4-b0de-36bfc0585afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:52.413105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 543/1085] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:04:52.711008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:04:52.712685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:52.737853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:52.879883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d466ae7e-4951-48a2-9502-2f03b307dc14 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:04:52.985001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:04:53.013976 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:04:53.014615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:04:53.015528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:04:53.015818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:04:53.017353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:04:53.120585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:04:53.122332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdf37a3-9c7f-445d-b833-e86cdbddf919 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:04:53.143211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 543/1086] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:04:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:02.411363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=68 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:03.140270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=54 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:06.033217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424071-275d-457f-9436-dd646eaa67a2 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:06.033781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424071-275d-457f-9436-dd646eaa67a2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:06.033781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1424071-275d-457f-9436-dd646eaa67a2 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:06.035071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 544/1087] 10.222.0.23 () {54 vars in 918 bytes} [Sun Oct 23 02:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:06.040275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818247c-81eb-498f-a472-247a01904f89 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:06.040684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818247c-81eb-498f-a472-247a01904f89 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:06.040821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818247c-81eb-498f-a472-247a01904f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:06.117178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6818247c-81eb-498f-a472-247a01904f89 None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:06.118373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6818247c-81eb-498f-a472-247a01904f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:06.222946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 544/1088] 10.222.0.23 () {58 vars in 1015 bytes} [Sun Oct 23 02:05:06 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:06.237558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f148ac3-41b0-4c4f-a64a-3158a2d2a86b None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:06.237843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f148ac3-41b0-4c4f-a64a-3158a2d2a86b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:06.238118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f148ac3-41b0-4c4f-a64a-3158a2d2a86b None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:06.239489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 545/1089] 10.222.0.23 () {54 vars in 918 bytes} [Sun Oct 23 02:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:06.243978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e493c8-ceea-42f4-83fa-5f458dccd486 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:06.244258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e493c8-ceea-42f4-83fa-5f458dccd486 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:06.244528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e493c8-ceea-42f4-83fa-5f458dccd486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:06.317125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:06.319609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e8e493c8-ceea-42f4-83fa-5f458dccd486 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:06.321310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e8e493c8-ceea-42f4-83fa-5f458dccd486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:06.437677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 545/1090] 10.222.0.23 () {58 vars in 1015 bytes} [Sun Oct 23 02:05:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:09.019502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:09.020006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:09.020006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:09.021286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 546/1091] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:09.026841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81f4d3-a6c5-4792-9f54-85a721098266 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:09.027236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81f4d3-a6c5-4792-9f54-85a721098266 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:09.027393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e81f4d3-a6c5-4792-9f54-85a721098266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:09.095708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8e81f4d3-a6c5-4792-9f54-85a721098266 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:09.096914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8e81f4d3-a6c5-4792-9f54-85a721098266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:09.150735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:09.192204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 546/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:09.483535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:09.486000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:09.630615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a1c9a-139c-4677-a94d-3ed48f4049dd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:09.705250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:09.744759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:09.745297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:09.745577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:09.745849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:09.747989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:09.836935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:09.838568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc58bd3-1fd6-4cd4-bb52-e1044322affb service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:09.856209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 547/1093] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:10.998411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:10.998966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:10.998966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:11.000262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 547/1094] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:11.005223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68f82-98ce-46f2-ae7d-e9171d11d068 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:11.005660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68f82-98ce-46f2-ae7d-e9171d11d068 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:11.005762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa68f82-98ce-46f2-ae7d-e9171d11d068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:11.080433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2aa68f82-98ce-46f2-ae7d-e9171d11d068 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:11.081630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2aa68f82-98ce-46f2-ae7d-e9171d11d068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:11.109756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:11.179165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 548/1095] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:11.327972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:11.329659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:11.475371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27729283-c872-4aa4-8974-e46a49c7ee92 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:11.515263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:11.587541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:11.588067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:11.588347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:11.588621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:11.590210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:11.679070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:11.680671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ece930-6078-473d-8820-bf16f37e398a service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:11.699498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 548/1096] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:12.812017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:12.812568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:12.812695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:12.814238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 549/1097] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:12.819095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c320-c2ac-4bc9-b304-e5fe9245f013 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:12.819503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c320-c2ac-4bc9-b304-e5fe9245f013 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:12.819662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99c320-c2ac-4bc9-b304-e5fe9245f013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:12.873153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:12.896318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ec99c320-c2ac-4bc9-b304-e5fe9245f013 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:12.897508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ec99c320-c2ac-4bc9-b304-e5fe9245f013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:12.993013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 549/1098] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:13.146234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:13.147791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:13.291383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c39b4d-da0e-4fd7-9d11-17bef3b1db1e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:13.304070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:13.403031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:13.404639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:13.404918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:13.405183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:13.406774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:13.496294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:13.497949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5743352-a303-4f1e-8f66-207b0d12cd9b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:13.515659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 550/1099] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:14.627161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:14.627761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:14.627761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:14.629050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 550/1100] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:14.633897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5034b-14a1-4895-a925-f815ebba658e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:14.634321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5034b-14a1-4895-a925-f815ebba658e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:14.634469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5034b-14a1-4895-a925-f815ebba658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:14.655326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:14.711842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1dd5034b-14a1-4895-a925-f815ebba658e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:14.713040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1dd5034b-14a1-4895-a925-f815ebba658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:14.808306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 551/1101] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:14.960678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:14.962102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:15.071347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:15.107978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df9c008-fbb8-486d-b734-ecb6dc815c4d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:15.219497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5215a210-644e-46a9-9986-66618a99064d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:15.220018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215a210-644e-46a9-9986-66618a99064d service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:15.220299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215a210-644e-46a9-9986-66618a99064d service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:15.220572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5215a210-644e-46a9-9986-66618a99064d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:15.222161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5215a210-644e-46a9-9986-66618a99064d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:15.299467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:15.313812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215a210-644e-46a9-9986-66618a99064d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:15.315422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5215a210-644e-46a9-9986-66618a99064d service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:15.333411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 551/1102] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:16.443937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:16.444506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:16.444506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:16.445889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 552/1103] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:16.451674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae70be-e084-47fd-b266-d96fda05cefa None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:16.452119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae70be-e084-47fd-b266-d96fda05cefa None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:16.452217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ae70be-e084-47fd-b266-d96fda05cefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:16.528219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f5ae70be-e084-47fd-b266-d96fda05cefa None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:16.529550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f5ae70be-e084-47fd-b266-d96fda05cefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:16.627076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 552/1104] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:16.778933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:16.780999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:16.864350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:16.925583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e891ca-c246-4a3f-aae6-d69b567abeb1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:17.039750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:17.040303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:17.040582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:17.040855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:17.042433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:17.091758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:17.135808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:17.137394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188d8943-5244-4142-b16f-dfa244245b40 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:17.155653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 553/1105] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:18.217615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9453-9728-441c-9684-c58a05436359 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:18.218218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9453-9728-441c-9684-c58a05436359 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:18.218218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e9453-9728-441c-9684-c58a05436359 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:18.219536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 553/1106] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:05:18.224645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff470ad-4f6b-4f55-9ae8-c5254e3919bd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:18.225058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff470ad-4f6b-4f55-9ae8-c5254e3919bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:18.225191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff470ad-4f6b-4f55-9ae8-c5254e3919bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:18.299960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7ff470ad-4f6b-4f55-9ae8-c5254e3919bd None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:18.301156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7ff470ad-4f6b-4f55-9ae8-c5254e3919bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:18.398849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 554/1107] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:18.552745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e9453-9728-441c-9684-c58a05436359 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:18.554260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-439e9453-9728-441c-9684-c58a05436359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:18.603227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:18.699211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e9453-9728-441c-9684-c58a05436359 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:18.816035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:18.816634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:18.816910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:18.817183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:18.818786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:18.826105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:18.912075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:18.913670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78fb52d-3a42-4d92-a5cc-3de221d41a1b service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:18.931345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 554/1108] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:19.977137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:19.977667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:19.977667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:19.978977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 555/1109] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:05:19.983863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e853-5ef5-44c7-9794-3753fc71fbc7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:19.984316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e853-5ef5-44c7-9794-3753fc71fbc7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:19.984451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd0e853-5ef5-44c7-9794-3753fc71fbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:20.058827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1bd0e853-5ef5-44c7-9794-3753fc71fbc7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:20.060581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1bd0e853-5ef5-44c7-9794-3753fc71fbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:20.129855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:20.161557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 555/1110] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:20.313356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:20.314801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:20.332392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:20.463200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0afbb10-3ef7-4793-bd2f-e5d728f3ee5e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:20.549559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:20.575398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:20.575931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:20.576408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:20.576543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:20.578159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:20.666078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:20.667689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb70bc5-b7d0-49f1-9c28-2863a43f1e6b service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:20.686636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 556/1111] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:21.733091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:21.733575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:21.733694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:21.735113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 556/1112] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:21.739867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98cb7-a919-493a-9fe3-aca65eddfa51 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:21.740295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98cb7-a919-493a-9fe3-aca65eddfa51 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:21.740398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff98cb7-a919-493a-9fe3-aca65eddfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:21.816314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0ff98cb7-a919-493a-9fe3-aca65eddfa51 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:21.817503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0ff98cb7-a919-493a-9fe3-aca65eddfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:21.859066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:21.913461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 557/1113] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:22.065280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:22.066773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:22.216482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13974c81-082b-4781-b70a-a86ca4ede80e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:22.270276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:22.329824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:22.330350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:22.330653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:22.330922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:22.332480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:22.420329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:22.421899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6869d8f8-7887-4ed0-8f17-51c51848a71d service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:22.440006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 557/1114] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:23.496416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:23.496955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:23.496955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:23.498306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 558/1115] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:23.503382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c490a-fff1-4e92-9376-f9da29015d96 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:23.503819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c490a-fff1-4e92-9376-f9da29015d96 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:23.503940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c490a-fff1-4e92-9376-f9da29015d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:23.579766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f53c490a-fff1-4e92-9376-f9da29015d96 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:23.580961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f53c490a-fff1-4e92-9376-f9da29015d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:23.582987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:23.677779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 558/1116] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:05:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:23.828474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:23.829898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:23.975399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5ba1a2-9aa6-40e6-b652-b4cd2dbe76e6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:24.003673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:24.089315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:24.089876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:24.090158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:24.090433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:24.092048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:24.181026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:24.182653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf70313c-5b52-4bbd-9c3c-a309564d0246 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:24.200853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 559/1117] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:26.289405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:26.290022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:26.290022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:26.291382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 559/1118] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:26.297620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b3f69-03ba-40d6-9cc0-194e87e8544a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:26.298063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b3f69-03ba-40d6-9cc0-194e87e8544a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:26.298162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b3f69-03ba-40d6-9cc0-194e87e8544a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:26.339392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:26.365195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-645b3f69-03ba-40d6-9cc0-194e87e8544a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:26.366381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-645b3f69-03ba-40d6-9cc0-194e87e8544a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:26.467125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 560/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:26.763314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:26.764880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:26.866747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e03ec6-4e68-4f5c-96e0-72682ee373f5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:26.899260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:26.941210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:26.941725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:26.942000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:26.942270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:26.943863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:27.051027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:27.052439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fba338-30bb-4b45-a619-304766b5925d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:27.071144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 560/1120] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:29.387193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:29.387781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:29.388061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:29.389428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 561/1121] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:29.394684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c0b-7d4b-4606-8ff5-0912a08f8fdf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:29.395124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c0b-7d4b-4606-8ff5-0912a08f8fdf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:29.395312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8677c0b-7d4b-4606-8ff5-0912a08f8fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:29.460382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e8677c0b-7d4b-4606-8ff5-0912a08f8fdf None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:29.461589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e8677c0b-7d4b-4606-8ff5-0912a08f8fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:29.474255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:29.564029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 561/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:29.853784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:29.855940 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:29.957560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78795849-ea0b-4df5-b82b-8322d26eb076 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:29.970680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:30.031450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:30.032049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:30.032329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:30.032600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:30.034166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:30.142384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:30.143848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dbb457-6b96-4981-9bac-d7286ec12f0b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:30.161424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 562/1123] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:32.640401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:32.640967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:32.640967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:32.642280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 562/1124] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:32.647746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af571-eccb-43d6-b3bc-e692d69b8d44 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:32.648191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af571-eccb-43d6-b3bc-e692d69b8d44 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:32.648286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669af571-eccb-43d6-b3bc-e692d69b8d44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:32.696727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:32.716336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-669af571-eccb-43d6-b3bc-e692d69b8d44 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:32.717542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-669af571-eccb-43d6-b3bc-e692d69b8d44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:32.814399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 563/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:33.103716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:33.105741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:33.206851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726c8b9e-071c-47a1-8ae4-57c60e7480c8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:33.247560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:33.281247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:33.281839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:33.282113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:33.282381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:33.283963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:33.391565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:33.392986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93480c4-b6b9-4abf-a049-cbec3824ff05 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:33.410803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 563/1126] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:35.715447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:35.715962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:35.716073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:35.717342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 564/1127] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:05:35.722997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555035af-9239-4359-817f-061992b67c9a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:35.723441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555035af-9239-4359-817f-061992b67c9a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:35.723552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555035af-9239-4359-817f-061992b67c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:35.788305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-555035af-9239-4359-817f-061992b67c9a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:35.789498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-555035af-9239-4359-817f-061992b67c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:35.809882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:35.884665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 564/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:36.169291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:36.170843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:36.271914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4311734-0222-4fab-8b37-34108fb548f3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:36.292922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:36.346944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:36.347544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:36.347860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:36.348121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:36.349700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:36.459315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:36.460655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c4929c-94d3-4703-9c2e-49e1594f1f6f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:36.478890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 565/1129] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:38.696691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:38.697197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:38.697296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:38.698812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 565/1130] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:38.704450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b59d-310f-45ac-87f6-1603e7a9ef5c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:38.704877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b59d-310f-45ac-87f6-1603e7a9ef5c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:38.704990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869b59d-310f-45ac-87f6-1603e7a9ef5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:38.769118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:38.772041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7869b59d-310f-45ac-87f6-1603e7a9ef5c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:38.773842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7869b59d-310f-45ac-87f6-1603e7a9ef5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:38.872061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 566/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:39.161693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:39.163291 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:39.265925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7256c8-fdad-483b-bede-9c36d493df5a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:39.314384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:39.339089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:39.339677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:39.339951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:39.340213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:39.341772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:39.447765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:39.449177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea131c96-4602-498f-9ed6-8aab87cbba1b service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:39.467320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 566/1132] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:41.830961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:41.831539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:41.831539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:41.832836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 567/1133] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:41.839126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18068b6-2c52-4c1a-881a-8ad144b4072e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:41.839578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18068b6-2c52-4c1a-881a-8ad144b4072e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:41.839801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18068b6-2c52-4c1a-881a-8ad144b4072e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:41.906777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a18068b6-2c52-4c1a-881a-8ad144b4072e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:41.907978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a18068b6-2c52-4c1a-881a-8ad144b4072e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:41.937131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:42.003990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 567/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:42.292770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:42.294283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:42.397561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299da630-9fdc-43e6-821b-5e703a42e38e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:42.425941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:42.472915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:42.473527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:42.473807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:42.474077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:42.475694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:42.587536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:42.589297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142e1b95-09af-451f-bb03-82103964a348 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:42.607853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 568/1135] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:44.807497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:44.808044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:44.808044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:44.809373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 568/1136] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:44.814695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978e8c-db13-443b-9b94-3eb49a89de6f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:44.815119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978e8c-db13-443b-9b94-3eb49a89de6f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:44.815256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf978e8c-db13-443b-9b94-3eb49a89de6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:44.880556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bf978e8c-db13-443b-9b94-3eb49a89de6f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:44.881744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bf978e8c-db13-443b-9b94-3eb49a89de6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:44.890086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:44.977427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 569/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:45.260929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:45.262468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:45.365587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c57068-3d0d-4076-aa14-e8a037a0081e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:45.421908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:45.440781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:45.441377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:45.441650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:45.441917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:45.443505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:45.552872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:45.554216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa70b005-0fb3-4e22-ac0f-2ae132cf1885 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:45.573777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 569/1138] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:47.789002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:47.789537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:47.789537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:47.790945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 570/1139] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:47.796584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551bff7-bb5e-42c1-af64-a6e18b80ac9c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:47.796988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551bff7-bb5e-42c1-af64-a6e18b80ac9c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:47.797129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1551bff7-bb5e-42c1-af64-a6e18b80ac9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:47.862074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1551bff7-bb5e-42c1-af64-a6e18b80ac9c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:47.863333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1551bff7-bb5e-42c1-af64-a6e18b80ac9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:47.901410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:47.962747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 570/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:48.248548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:48.250073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:48.352300 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7954-f6f7-41e8-885b-48b0cf74f492 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:48.391046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:48.426361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:48.426990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:48.427273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:48.427550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:48.429127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:48.537998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:48.539378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9ef240-320a-4eee-a1dd-2091463170b5 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:48.558754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 571/1141] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:50.761328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:50.761880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:50.761880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:50.763288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 571/1142] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:50.769217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af3ad-9761-4f62-b79c-c36e7a6fa3e1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:50.769217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af3ad-9761-4f62-b79c-c36e7a6fa3e1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:50.769671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365af3ad-9761-4f62-b79c-c36e7a6fa3e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:50.834559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-365af3ad-9761-4f62-b79c-c36e7a6fa3e1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:50.835742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-365af3ad-9761-4f62-b79c-c36e7a6fa3e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:50.853466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:50.931734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 572/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:51.219873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:51.221381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:51.323284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f37726e-40ad-4f2a-9a37-7174cc378773 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:51.389612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:51.396268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:51.396861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:51.397136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:51.397404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:51.398998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:51.508421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:51.510279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618be26d-741f-47ff-80f8-1fc6276fc1f5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:51.528432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 572/1144] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:05:53.708997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:53.709472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:53.709580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:53.710986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 573/1145] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:05:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:05:53.716398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e70b4-34be-459a-b4cd-bb1482077665 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:53.716834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e70b4-34be-459a-b4cd-bb1482077665 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:53.716939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8e70b4-34be-459a-b4cd-bb1482077665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:53.781916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ea8e70b4-34be-459a-b4cd-bb1482077665 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:05:53.783126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea8e70b4-34be-459a-b4cd-bb1482077665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:53.830248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:53.877949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 573/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:05:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:05:54.161723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:05:54.163358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:54.265411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a110c9-462b-4dee-938e-fd98f95cc8af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:05:54.312020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:05:54.338640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:05:54.339234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:05:54.339511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:05:54.339777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:05:54.341331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:05:54.450200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:05:54.451863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950b7e7-3d58-4fe6-8ef5-dbcde296a6f1 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:05:54.469319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 574/1147] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:05:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:03.876457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=22 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:04.467096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=66 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:14.407681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52028d-96c0-4edf-8ee8-5a09243f3d92 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:14.408246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52028d-96c0-4edf-8ee8-5a09243f3d92 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:14.408246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e52028d-96c0-4edf-8ee8-5a09243f3d92 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:14.409589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 574/1148] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:14.415156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbb819-4446-453f-8b62-e2e3358354be None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:14.415587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbb819-4446-453f-8b62-e2e3358354be None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:14.415715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edbb819-4446-453f-8b62-e2e3358354be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:14.482349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4edbb819-4446-453f-8b62-e2e3358354be None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:14.483577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4edbb819-4446-453f-8b62-e2e3358354be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:14.578456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 575/1149] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:06:14.877521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:14.877807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:14.878083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:14.879444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 575/1150] 10.222.0.23 () {54 vars in 962 bytes} [Sun Oct 23 02:06:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:14.884781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc09e6-79a3-4353-9a0b-7566f827b73f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:14.885083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc09e6-79a3-4353-9a0b-7566f827b73f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:14.885361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbc09e6-79a3-4353-9a0b-7566f827b73f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:14.962105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2fbc09e6-79a3-4353-9a0b-7566f827b73f None None] MFA Rules not processed for user `19022fccab844d1dac60f5eddf67f323`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:14.963268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2fbc09e6-79a3-4353-9a0b-7566f827b73f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:14.983982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:15.039944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 576/1151] 10.222.0.23 () {58 vars in 1059 bytes} [Sun Oct 23 02:06:14 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:15.047698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:15.049306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:15.151716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464cfa3-b526-4eef-ac0c-5b44db9e7fbf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:15.223866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:15.224388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:15.224667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:15.224938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:15.227084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:15.279634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:15.333922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:15.336299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bbb52a-c580-4aed-afe7-ff49139278bd service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:15.356169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 576/1152] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:17.601538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:17.602062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:17.602062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:17.603408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 577/1153] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:17.608817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c3bfc-fafa-4194-8bcd-172788f18154 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:17.609230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c3bfc-fafa-4194-8bcd-172788f18154 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:17.609362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004c3bfc-fafa-4194-8bcd-172788f18154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:17.674357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-004c3bfc-fafa-4194-8bcd-172788f18154 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:17.675577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-004c3bfc-fafa-4194-8bcd-172788f18154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:17.771275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 577/1154] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:18.057667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:18.059278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:18.185265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cfee41-3eac-4718-aa5c-3b120cba95a4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:18.236869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:18.281373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:18.281971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:18.282247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:18.282607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:18.283238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:18.284321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:06:18.284321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:06:18.284321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:06:18.285447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7feb2c-36a0-4eb0-8c14-c51109d30cfc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:18.304801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 578/1155] 10.222.0.23 () {56 vars in 1108 bytes} [Sun Oct 23 02:06:18 2022] GET /identity/v3/projects => generated 2553 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:06:20.513528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:20.514148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:20.514148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:20.515466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 578/1156] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:20.520839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee70df-90b8-4ee3-a533-2b4a61260efd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:20.521287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee70df-90b8-4ee3-a533-2b4a61260efd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:20.521395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ee70df-90b8-4ee3-a533-2b4a61260efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:20.588650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-66ee70df-90b8-4ee3-a533-2b4a61260efd None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:20.589842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66ee70df-90b8-4ee3-a533-2b4a61260efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:20.687058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 579/1157] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:06:20.988525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:20.990045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:20.999748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:21.133378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43da1b8e-da16-470e-8741-7727ba23433a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:21.213210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:21.245345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:21.245946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:21.246221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:21.246521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:21.248127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:21.337591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:21.339224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bb3b9a-c70b-434c-9b21-6c8dfb916ae4 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:21.357089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 579/1158] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:23.685292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:23.685676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:23.685676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:23.687016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 580/1159] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:23.692459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520995b-4512-42cc-b19d-1ab5745e96ea None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:23.692857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520995b-4512-42cc-b19d-1ab5745e96ea None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:23.693007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4520995b-4512-42cc-b19d-1ab5745e96ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:23.759828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4520995b-4512-42cc-b19d-1ab5745e96ea None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:23.761012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4520995b-4512-42cc-b19d-1ab5745e96ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:23.806137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:23.856543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 580/1160] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:24.142371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:24.143927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:24.168515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:24.247895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a690f66c-fb29-432a-aef7-74306fcdabaf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:24.320280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:24.320884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:24.321165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:24.321440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:24.323049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:24.400705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:24.435919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:24.437364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca095683-2eb9-4a87-9652-26069b3dd93a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:24.456075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 581/1161] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:06:26.652639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:26.653210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:26.653324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:26.654730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 581/1162] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:26.659975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782518-9c70-4908-8e9f-790f13f7baeb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:26.660389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782518-9c70-4908-8e9f-790f13f7baeb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:26.660523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85782518-9c70-4908-8e9f-790f13f7baeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:26.726126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-85782518-9c70-4908-8e9f-790f13f7baeb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:26.727344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-85782518-9c70-4908-8e9f-790f13f7baeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:26.824681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 582/1163] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:27.110668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:27.112232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:27.215613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a508f-967b-4862-b6cc-da23803ae7d8 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:27.285372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:27.287416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:27.288096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:27.288384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:27.288847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:27.290825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:27.397949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:27.399392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a226ea-8714-41d9-9a8a-f91aa151ca91 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:27.417507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 582/1164] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:29.732322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:29.732857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:29.732857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:29.734286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 583/1165] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:29.740101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fd9-ed1a-47b8-8fdd-f692b2e22a5c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:29.740521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fd9-ed1a-47b8-8fdd-f692b2e22a5c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:29.740639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cf9fd9-ed1a-47b8-8fdd-f692b2e22a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:29.806282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e4cf9fd9-ed1a-47b8-8fdd-f692b2e22a5c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:29.807510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4cf9fd9-ed1a-47b8-8fdd-f692b2e22a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:29.858930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:29.903108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 583/1166] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:30.193799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:30.195372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:30.231334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:30.299835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7c8fd-3e69-4790-b090-a0e608e3ec19 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:30.374042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:30.374666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:30.374953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:30.375214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:30.376775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:30.458993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:30.487763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:30.489183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c56411-948b-4d6a-8cc3-f9d369427a8f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:30.507378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 584/1167] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:32.907968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:32.908497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:32.908497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:32.909843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 584/1168] 10.222.0.23 () {54 vars in 927 bytes} [Sun Oct 23 02:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:06:32.915327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffeda-129f-492f-8ff2-0b5443592245 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:32.915772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffeda-129f-492f-8ff2-0b5443592245 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:32.915877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fffeda-129f-492f-8ff2-0b5443592245 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:32.981745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-29fffeda-129f-492f-8ff2-0b5443592245 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:06:32.982926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29fffeda-129f-492f-8ff2-0b5443592245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:33.079572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 585/1169] 10.222.0.23 () {58 vars in 1024 bytes} [Sun Oct 23 02:06:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:06:33.369068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:06:33.370655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:33.470808 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c2b32-c0f0-4189-8c24-2e8be0950f59 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:06:33.542648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:06:33.543240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:06:33.543518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:06:33.543789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:06:33.545355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:06:33.553694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:33.651687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:06:33.653039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b01df5e-12b4-4cce-8eac-aca8672de060 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:06:33.671240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 585/1170] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:06:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:06:43.077984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=82 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:06:43.669005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=50 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:46.834137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711929-220e-429d-925f-9ed6761d0639 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:46.834735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711929-220e-429d-925f-9ed6761d0639 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:46.834735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16711929-220e-429d-925f-9ed6761d0639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:46.911516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-16711929-220e-429d-925f-9ed6761d0639 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:07:46.912714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16711929-220e-429d-925f-9ed6761d0639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.008860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 586/1171] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:07:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:07:47.016694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.017178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.017262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.093324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:07:47.094562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.192752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 586/1172] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:07:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:07:47.197633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16711929-220e-429d-925f-9ed6761d0639 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.198876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16711929-220e-429d-925f-9ed6761d0639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.210228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16711929-220e-429d-925f-9ed6761d0639 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:47.271734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:47.305992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:47.306698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.306978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.307251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.307883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:07:47.308182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:47.309194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:07:47.309194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:07:47.309194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:07:47.310629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:47.321748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 587/1173] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:07:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:07:47.332554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.341821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a290594e-f26b-4a03-ae97-5c6610c99975 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:47.402405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:47.435747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:47.436355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.436634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.436907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.437531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:47.438658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:07:47.438658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 02:07:47.438658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 02:07:47.439847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:47.494634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 587/1174] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:07:47 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:07:47.499670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.501193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.534237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293ecd4-f5e7-4ef8-a1dd-75dc17669288 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:47.626578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:47.627175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.627450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.627718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.628345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:47.629335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:07:47.629335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:07:47.629335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:07:47.630530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:47.672386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 588/1175] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:07:47 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:07:47.676504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.678000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.740508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e5996c-4fb6-460f-aec0-7c7d81db95f6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:47.822028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:47.834465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:47.835090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.835371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.835643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.836278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:47.837891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:47.850248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 588/1176] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:07:47.854235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.855823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:47.867966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480312f8-fe3d-465e-a2f5-053e0f70ff70 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:47.893667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:47.967508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:47.968116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:47.968391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:47.968663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:47.969293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:47.970914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8769c398-f284-48bd-9938-5a3d2d360963 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:47.983018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 589/1177] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:07:47.987166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:47.988707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.022394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826fb97c-3bbc-4387-9815-53fda321aa09 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:48.116593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:48.117244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.117519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.117787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] PATH_INFO: `/v3/projects/5b7be407f66445b991bff9b4fa44681d/users/9f492c70feaa4519a2ef7c689062b8f0/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.141087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b7be407f66445b991bff9b4fa44681d, user_id=9f492c70feaa4519a2ef7c689062b8f0, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:48.143843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c93a61-1293-4458-acf3-8bc14a64894d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:48.167991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 589/1178] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:07:47 2022] PUT /identity/v3/projects/5b7be407f66445b991bff9b4fa44681d/users/9f492c70feaa4519a2ef7c689062b8f0/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:07:48.179347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da0d0-6df0-488b-a2f2-5532188c07b8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.179643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da0d0-6df0-488b-a2f2-5532188c07b8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.179974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1da0d0-6df0-488b-a2f2-5532188c07b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.237311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd1da0d0-6df0-488b-a2f2-5532188c07b8 None None] MFA Rules not processed for user `9f492c70feaa4519a2ef7c689062b8f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:07:48.238537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd1da0d0-6df0-488b-a2f2-5532188c07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.316306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 590/1179] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:07:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:07:48.330102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.330388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.330701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.332033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 590/1180] 10.222.0.23 () {54 vars in 963 bytes} [Sun Oct 23 02:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:07:48.336809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e892-473e-4530-a7b5-43746f0db9fd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.337100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e892-473e-4530-a7b5-43746f0db9fd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.337377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3e892-473e-4530-a7b5-43746f0db9fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.370033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:48.416920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-00b3e892-473e-4530-a7b5-43746f0db9fd None None] MFA Rules not processed for user `a323cd646bae44b5a6f63714b5cdf70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:07:48.418120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00b3e892-473e-4530-a7b5-43746f0db9fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.493928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 591/1181] 10.222.0.23 () {58 vars in 1060 bytes} [Sun Oct 23 02:07:48 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:07:48.502390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:48.503965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.553107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:48.606999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1c4049-e222-4849-94ed-29bccefb8850 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:48.678809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:48.679328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.679588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.679857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.681446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.784516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:48.785948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:48.789465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:48.804876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 591/1182] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:07:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:07:48.850961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf45b-f71a-4f15-8ff7-479235ed6495 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:48.851251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf45b-f71a-4f15-8ff7-479235ed6495 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:48.851528 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531bf45b-f71a-4f15-8ff7-479235ed6495 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:48.852819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 592/1183] 10.222.0.23 () {52 vars in 830 bytes} [Sun Oct 23 02:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:07:48.856352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:48.858320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:48.957493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ff08a8-8d8d-4ed9-989a-52b9fb746fb8 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:49.047789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:49.048468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:49.048669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:49.048939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] PATH_INFO: `/v3/users/9f492c70feaa4519a2ef7c689062b8f0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:49.061392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f492c70feaa4519a2ef7c689062b8f0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:49.062397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:07:49.062397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 02:07:49.062397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 02:07:49.063612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:49.073185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:49.131667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f492c70feaa4519a2ef7c689062b8f0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:49.145846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f492c70feaa4519a2ef7c689062b8f0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:49.155950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f492c70feaa4519a2ef7c689062b8f0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:49.156545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f492c70feaa4519a2ef7c689062b8f0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:49.156970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f492c70feaa4519a2ef7c689062b8f0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:49.185459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 592/1184] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:07:48 2022] DELETE /identity/v3/users/9f492c70feaa4519a2ef7c689062b8f0 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:07:49.189773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:49.190247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:49.190351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:49.246578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:07:49.247998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:49.321985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:49.334726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 593/1185] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:07:49.345352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:49.346614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:49.445309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68c652a-4f02-4ffd-a2ba-35b3089efe7d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:49.498957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:07:49.520291 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:49.520899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:49.521178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:49.521450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:49.523065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:49.631164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:49.632601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b74fead-c2fb-43a6-9fb3-b13c1ec15cc8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:49.651151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 593/1186] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:07:49.971992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:07:49.973243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:07:49.985047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0cfd02-8bcb-4a54-bbc9-3f3661b907fa None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:07:50.084442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:07:50.084986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:07:50.085264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:07:50.085532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] PATH_INFO: `/v3/projects/5b7be407f66445b991bff9b4fa44681d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:07:50.095309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b7be407f66445b991bff9b4fa44681d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:07:50.096318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:07:50.096318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:07:50.096318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:07:50.097498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:07:50.153840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b7be407f66445b991bff9b4fa44681d', 'request_id': 'req-c9178684-eb9f-4192-a597-497be5d59073'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:50.166776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b7be407f66445b991bff9b4fa44681d', 'request_id': 'req-c9178684-eb9f-4192-a597-497be5d59073'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:50.173186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b7be407f66445b991bff9b4fa44681d', 'request_id': 'req-c9178684-eb9f-4192-a597-497be5d59073'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:50.200320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] The token cache is being invalidate because project 5b7be407f66445b991bff9b4fa44681d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:07:50.200927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:50.201279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c9178684-eb9f-4192-a597-497be5d59073 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:07:50.202608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 594/1187] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:07:49 2022] DELETE /identity/v3/projects/5b7be407f66445b991bff9b4fa44681d => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:07:59.648913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=62 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:08:00.198266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:09:15.593952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:15.594575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:15.594575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:15.595881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 594/1188] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:15.601355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5b7b-854b-44ee-831a-3f9e2921faf2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:15.601773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5b7b-854b-44ee-831a-3f9e2921faf2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:15.601909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb5b7b-854b-44ee-831a-3f9e2921faf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:15.683922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-eddb5b7b-854b-44ee-831a-3f9e2921faf2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:15.685434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eddb5b7b-854b-44ee-831a-3f9e2921faf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:15.790553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 595/1189] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:09:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:15.948809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:15.950958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:16.072310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48570ba9-8fab-473e-8b59-fe8a04f129cf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:16.078006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:16.078668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:16.078952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:16.079230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:16.087477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:16.089049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beda6acc-d14b-4556-9451-5745419e293c service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:16.098263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 595/1190] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:17.169551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffe3ab-7e1f-4cd7-99dc-f2aa3444e630 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:17.170040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffe3ab-7e1f-4cd7-99dc-f2aa3444e630 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:17.170143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ffe3ab-7e1f-4cd7-99dc-f2aa3444e630 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:17.171488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 596/1191] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:17.176387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeebe6-e0c8-4dee-9c76-c65552ea70e1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:17.176815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeebe6-e0c8-4dee-9c76-c65552ea70e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:17.177644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aeeebe6-e0c8-4dee-9c76-c65552ea70e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:17.197491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5aeeebe6-e0c8-4dee-9c76-c65552ea70e1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:17.198731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5aeeebe6-e0c8-4dee-9c76-c65552ea70e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:17.206563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 596/1192] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:17.360222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffe3ab-7e1f-4cd7-99dc-f2aa3444e630 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:17.374502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ffe3ab-7e1f-4cd7-99dc-f2aa3444e630 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:17.379858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:17.380477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:17.380756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:17.381030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:17.389093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:17.390641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a8c2a7-9006-46f5-ad32-3d84ec34333e service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:17.400330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 597/1193] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:17.544559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a031-ab62-40b5-b605-cca08196b08f None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:17.544863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a031-ab62-40b5-b605-cca08196b08f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:17.545137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e67a031-ab62-40b5-b605-cca08196b08f None None] PATH_INFO: `/v3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:17.546527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 597/1194] 10.222.0.49 () {54 vars in 925 bytes} [Sun Oct 23 02:09:17 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:09:17.553599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220f42-8b66-4b47-99be-7235fde1276b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:17.553910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220f42-8b66-4b47-99be-7235fde1276b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:17.554186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d220f42-8b66-4b47-99be-7235fde1276b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:17.613735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5d220f42-8b66-4b47-99be-7235fde1276b None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:17.615174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d220f42-8b66-4b47-99be-7235fde1276b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:17.632109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:09:17.687719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 598/1195] 10.222.0.49 () {58 vars in 1015 bytes} [Sun Oct 23 02:09:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:26.562628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e044b-7d30-4a71-a800-3cf95ff0f25d None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:26.563064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e044b-7d30-4a71-a800-3cf95ff0f25d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:26.563168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e044b-7d30-4a71-a800-3cf95ff0f25d None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:26.564510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 598/1196] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:26.569848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed86c-b545-4a55-9687-c8ac1147c6bd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:26.570321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed86c-b545-4a55-9687-c8ac1147c6bd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:26.570431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eed86c-b545-4a55-9687-c8ac1147c6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:26.590214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c9eed86c-b545-4a55-9687-c8ac1147c6bd None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:26.591462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c9eed86c-b545-4a55-9687-c8ac1147c6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:26.600947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 599/1197] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:09:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:26.608649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e044b-7d30-4a71-a800-3cf95ff0f25d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:26.621949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e044b-7d30-4a71-a800-3cf95ff0f25d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:26.626854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:26.627442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:26.627724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:26.628000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:26.636991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:26.638596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ffa6a-b510-4bda-b850-6e243f96c60d service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:26.648921 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 599/1198] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:28.895878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6fa3b-753f-418f-b898-758c1c9157cb None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:28.896418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6fa3b-753f-418f-b898-758c1c9157cb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:28.896418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa6fa3b-753f-418f-b898-758c1c9157cb None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:28.897825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 600/1199] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:09:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:28.902932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa4c-c01b-4422-9e87-f3a2811d5c96 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:28.903308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa4c-c01b-4422-9e87-f3a2811d5c96 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:28.903479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223aa4c-c01b-4422-9e87-f3a2811d5c96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:28.922701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-8223aa4c-c01b-4422-9e87-f3a2811d5c96 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:28.923927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8223aa4c-c01b-4422-9e87-f3a2811d5c96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:28.931914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 600/1200] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:09:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:29.087841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6fa3b-753f-418f-b898-758c1c9157cb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:29.102416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa6fa3b-753f-418f-b898-758c1c9157cb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:29.107791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:29.108383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:29.108663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:29.108937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:29.117344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:29.118938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd36e18-a35a-4cb7-98c9-584ddb8ae3ae service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:29.127979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 601/1201] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:09:30.185036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa231819-21ce-4840-91d2-3bf2ed30af21 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:30.185869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa231819-21ce-4840-91d2-3bf2ed30af21 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:30.185869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa231819-21ce-4840-91d2-3bf2ed30af21 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:30.187246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 601/1202] 127.0.0.1 () {54 vars in 901 bytes} [Sun Oct 23 02:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:30.193031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c22e8-efd9-4ff0-a989-677bace9b310 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:30.193466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c22e8-efd9-4ff0-a989-677bace9b310 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:30.193564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193c22e8-efd9-4ff0-a989-677bace9b310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:30.212969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-193c22e8-efd9-4ff0-a989-677bace9b310 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:30.214159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-193c22e8-efd9-4ff0-a989-677bace9b310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:30.221907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 602/1203] 127.0.0.1 () {58 vars in 998 bytes} [Sun Oct 23 02:09:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:30.376045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa231819-21ce-4840-91d2-3bf2ed30af21 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:30.390629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa231819-21ce-4840-91d2-3bf2ed30af21 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:30.396812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:30.397394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:30.397668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:30.397934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:30.405740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:30.407327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf62cf1-9671-461e-95ac-9c5862b1ef23 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:30.416741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 602/1204] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:34.373080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:34.373621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:34.373621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:34.375021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 603/1205] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:34.380797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5e30-f148-4db7-8b82-ef7d5072037a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:34.381230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5e30-f148-4db7-8b82-ef7d5072037a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:34.381354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641d5e30-f148-4db7-8b82-ef7d5072037a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:34.401475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-641d5e30-f148-4db7-8b82-ef7d5072037a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:34.402978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-641d5e30-f148-4db7-8b82-ef7d5072037a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:34.412325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 603/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:09:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:34.703968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:34.705698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:34.783502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4306605c-5e14-46bf-853a-7706f5e24574 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:34.787169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:34.787769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:34.788050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:34.788325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:34.797881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:34.799322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727163b7-af1e-4751-b61b-a1e3a81cbb6b service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:34.809430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 604/1207] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:36.938874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e166-0b8c-4ec5-9a11-21a36dc10bc5 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:36.939365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e166-0b8c-4ec5-9a11-21a36dc10bc5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:36.939365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d750e166-0b8c-4ec5-9a11-21a36dc10bc5 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:36.940706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 604/1208] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:09:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:36.946421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86d69-1656-4b3c-ab19-7db7e8cba566 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:36.946842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86d69-1656-4b3c-ab19-7db7e8cba566 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:36.947002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae86d69-1656-4b3c-ab19-7db7e8cba566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:36.966962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bae86d69-1656-4b3c-ab19-7db7e8cba566 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:36.968152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bae86d69-1656-4b3c-ab19-7db7e8cba566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:36.976177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 605/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:09:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:37.262939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e166-0b8c-4ec5-9a11-21a36dc10bc5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:37.273607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d750e166-0b8c-4ec5-9a11-21a36dc10bc5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:37.277311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:37.277903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:37.278181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:37.278479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:37.287867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:37.289329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c830663-8b6e-430a-abd2-9464582e24a1 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:37.298639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 605/1210] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:40.950141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:40.950694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:40.950694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:40.952022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 606/1211] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:40.958052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ae3fa-f7f2-478e-85c4-e01038a25d96 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:40.958580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ae3fa-f7f2-478e-85c4-e01038a25d96 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:40.958682 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07ae3fa-f7f2-478e-85c4-e01038a25d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:40.966152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:09:40.978849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e07ae3fa-f7f2-478e-85c4-e01038a25d96 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:40.980042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e07ae3fa-f7f2-478e-85c4-e01038a25d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:40.987807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 606/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:09:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:41.289071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:41.290818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:41.355513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:09:41.369032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a86d1-b7f7-4c3f-a83e-f0a2530bff7d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:41.372630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:41.373227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:41.373515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:41.373790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:41.382765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:41.384192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d307c78-e122-4c91-88ea-4535d9bcef13 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:41.393459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 607/1213] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:43.606602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:43.607260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:43.607260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:43.608519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 607/1214] 127.0.0.1 () {54 vars in 917 bytes} [Sun Oct 23 02:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:09:43.614233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d67bb-039a-4c01-bdf9-b874210024eb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:43.614706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d67bb-039a-4c01-bdf9-b874210024eb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:43.614811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066d67bb-039a-4c01-bdf9-b874210024eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:43.635200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-066d67bb-039a-4c01-bdf9-b874210024eb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:09:43.636361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-066d67bb-039a-4c01-bdf9-b874210024eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:43.643985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 608/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Sun Oct 23 02:09:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:09:43.955859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:09:43.958049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:09:43.970979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de5ec98-befb-42bf-a531-f6c7e4691e51 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:09:43.975020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:09:43.975637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:09:43.975920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:09:43.976195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:09:43.984945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:09:43.986351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f132e039-f6c0-4213-b1d3-09334a4dc039 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:09:43.996459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 608/1216] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:09:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:09:53.630357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=11 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:09:53.992384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=8 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:08.777270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:08.777933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:08.777933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:08.803549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:08.804775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:08.818497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 609/1217] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:08.828792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:08.829393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:08.829393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:08.863576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:08.864793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:08.877057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 609/1218] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:08.881872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:08.898087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfe3045-1e17-4987-8196-df6f1d67176e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:08.907125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:08.907879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:08.908329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:08.908707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:08.909669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:08.910118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:08.915630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9045714-8464-4e5c-a364-5e2b6728f43d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:08.940194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 610/1219] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:08.953416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:08.973071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f532bd-e2ba-43c0-b672-e1e56a167227 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:08.983454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:08.984756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:08.985621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:08.985895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:08.986574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:08.988483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.039922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd461-a475-40bc-9902-72cb30699583 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.040925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd461-a475-40bc-9902-72cb30699583 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.041672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bcd461-a475-40bc-9902-72cb30699583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.059806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 610/1220] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:08 2022] POST /identity/v3/projects => generated 355 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:09.064648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.074856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-78bcd461-a475-40bc-9902-72cb30699583 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.076500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-78bcd461-a475-40bc-9902-72cb30699583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.191297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 611/1221] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.193479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f580f7-e42c-4e59-bf4d-67da9ba63a4e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.205447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.205992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.206321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.206631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.207296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.209082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:11:09.209082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 02:11:09.209082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 02:11:09.210429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.225382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.226013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.226586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.260744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 611/1222] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/users => generated 278 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:09.261621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.263068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.270468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.277145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 612/1223] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.280283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275343dd-0429-496c-93eb-4b84c2b46ae6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.288439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.291795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.292426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.292836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.293103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.293929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.295951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.305965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5573534-bdf7-436b-9d6e-b2ec3e6c07c6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.314549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.315468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.315988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.316485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.320428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:09.320577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 612/1224] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:09.321185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.323467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca77ea83-4601-4f4b-aa90-e6b139823dcb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.323748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.341191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba20d788-806b-4bd0-bb73-6d3070ae404c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.347352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 613/1225] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:09.347905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.348732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.349016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.349291 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.349967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.350292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.352174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c7337a-39d0-48e8-be95-42d9b991a3a1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.352822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.353335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.378486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 613/1226] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:09.378486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.378486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.378486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.385873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.387560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.399749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 614/1227] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.400869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.424018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9d25b3-a0ca-41a2-9fac-7e7645a9aed6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.427360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.428846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.430208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.431208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.431783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.432295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] PATH_INFO: `/v3/projects/8eebb5bda00044c2b1c5bfa0078ab047/users/b99ef823f08140d69edb54a7826b4763/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.446276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 614/1228] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.446915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.450711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eebb5bda00044c2b1c5bfa0078ab047, user_id=b99ef823f08140d69edb54a7826b4763, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.454553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cb232d-c1f6-4b7c-9420-ddad24a92fb3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.467407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e551b5-3a7b-4a7b-a72f-67589f3cf110 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.476077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 615/1229] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:09 2022] PUT /identity/v3/projects/8eebb5bda00044c2b1c5bfa0078ab047/users/b99ef823f08140d69edb54a7826b4763/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:09.477788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796496a-3340-4650-8262-c243935e770a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.478394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796496a-3340-4650-8262-c243935e770a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.478955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6796496a-3340-4650-8262-c243935e770a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.483141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.484049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.484945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.485366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.486014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.488067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1713139a-91d8-4eb2-8b97-98e769c588ff admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.509288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6796496a-3340-4650-8262-c243935e770a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.511805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6796496a-3340-4650-8262-c243935e770a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.530188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 615/1230] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/projects => generated 375 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:09.532723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255660a-e1d1-4da4-8eb0-446c6ed35f88 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.533161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255660a-e1d1-4da4-8eb0-446c6ed35f88 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.533515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a255660a-e1d1-4da4-8eb0-446c6ed35f88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.554051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a255660a-e1d1-4da4-8eb0-446c6ed35f88 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.555302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a255660a-e1d1-4da4-8eb0-446c6ed35f88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.580631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 616/1231] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.581878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0da-9e45-4bb3-a977-8b430a80299b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.582194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0da-9e45-4bb3-a977-8b430a80299b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.582642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1c0da-9e45-4bb3-a977-8b430a80299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.601245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8eb1c0da-9e45-4bb3-a977-8b430a80299b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.602425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8eb1c0da-9e45-4bb3-a977-8b430a80299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.617126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 616/1232] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.618482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4a52a-7fd1-4ae8-8bda-e71c64c69d55 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.618758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4a52a-7fd1-4ae8-8bda-e71c64c69d55 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.619034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc4a52a-7fd1-4ae8-8bda-e71c64c69d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.626303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:09.630801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:09.647966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dfc4a52a-7fd1-4ae8-8bda-e71c64c69d55 None None] MFA Rules not processed for user `b99ef823f08140d69edb54a7826b4763`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.649174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dfc4a52a-7fd1-4ae8-8bda-e71c64c69d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.667618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 617/1233] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.668892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1fae-954d-4582-a693-ba20aeb4991a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.669176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1fae-954d-4582-a693-ba20aeb4991a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.669446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe1fae-954d-4582-a693-ba20aeb4991a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.690485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4bfe1fae-954d-4582-a693-ba20aeb4991a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.691692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4bfe1fae-954d-4582-a693-ba20aeb4991a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.699860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 618/1234] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.703732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.704130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.704518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.722669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 617/1235] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.723828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a571a-733b-4721-870c-0d218b688d0a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.723946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.724137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a571a-733b-4721-870c-0d218b688d0a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.724378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a571a-733b-4721-870c-0d218b688d0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.725027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.732769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 619/1236] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.733518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.743640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fedc2d2-54b9-450d-bf64-a95705b2477f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.748389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.748984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.749262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.749373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-214a571a-733b-4721-870c-0d218b688d0a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.749835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.750763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-214a571a-733b-4721-870c-0d218b688d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.750877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.752678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a64891-4b6a-47e1-8826-ae8bff54c473 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.758743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 618/1237] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.759582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a571a-733b-4721-870c-0d218b688d0a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.771200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a571a-733b-4721-870c-0d218b688d0a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.776391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.776995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.777276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.777700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.778336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:09.778484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 620/1238] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/users => generated 288 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:09.779004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.779674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d5e8-2ac9-4668-9703-2152d2f2efda None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.779785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:11:09.779785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 02:11:09.779785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 02:11:09.779975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d5e8-2ac9-4668-9703-2152d2f2efda None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.780206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2548d5e8-2ac9-4668-9703-2152d2f2efda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.780631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.800673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 619/1239] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:09.800673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.800970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2548d5e8-2ac9-4668-9703-2152d2f2efda None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.801821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2548d5e8-2ac9-4668-9703-2152d2f2efda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.805599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d50967-17ea-4c3d-a150-840d643ea133 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.809852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 621/1240] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.810179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.810801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.811040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559cbfd-0367-4e1c-a4e8-dabb84cf7bd2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.811145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.811327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559cbfd-0367-4e1c-a4e8-dabb84cf7bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.811443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.811620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7559cbfd-0367-4e1c-a4e8-dabb84cf7bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.811955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:09.812887 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.814491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.834519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 620/1241] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:09.834519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.838009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4663a0-63d2-4f96-9748-5725908c637e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.839668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7559cbfd-0367-4e1c-a4e8-dabb84cf7bd2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.840850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7559cbfd-0367-4e1c-a4e8-dabb84cf7bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.842702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.843340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.843735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.844017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.844632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.846541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.849182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 622/1242] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.850384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7160-b004-428a-88cb-f73835bd7f7f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.850691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7160-b004-428a-88cb-f73835bd7f7f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.850961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086c7160-b004-428a-88cb-f73835bd7f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.872477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-086c7160-b004-428a-88cb-f73835bd7f7f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.874388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-086c7160-b004-428a-88cb-f73835bd7f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.883574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 623/1243] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.886896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 621/1244] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/projects => generated 355 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:09.887047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.887159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.887239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.887239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.905234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:09.906208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:09.949069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f7a544-a0f0-4241-865e-bb8713c38219 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.954298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.954930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.955215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.955487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.956125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.957708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.966964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 624/1245] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:09.967895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.971176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 622/1246] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:09.972029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:09.978272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b880-d179-44c0-aece-18e5f5f6f584 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.981283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce7ba1b-290d-4583-b64d-da5d396620ac admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:09.982861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.983484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.983766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.984172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.984847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.985857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:11:09.985857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:11:09.985857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:11:09.986628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:09.987073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:09.987358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:09.987698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:09.988040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:09.988737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:09.989107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:09.990798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.004159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 623/1247] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:10.010646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.015314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18df8f8b-8e6a-4c96-8856-816b48347350 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.018933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 625/1248] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:09 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.018933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.035479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.076356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.077622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.078680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.079025 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.079719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.080846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff5dc21-800a-4b1e-8ec6-ad48b884cc6f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.081690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.085519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.086121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.086400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.086932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.087330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:10.087636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.089226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.091402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.102642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 626/1249] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:10.103609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.115014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9028c9cb-e515-400a-83eb-dbac0a216369 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.118179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 624/1250] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/projects => generated 393 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.119069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.120021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.120725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.121241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.121446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.122140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.124211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.149298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 627/1251] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 284 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.149298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.157649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d66b07-1706-4bae-a57a-4810361d5276 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.183120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c08201-2363-464b-bcb2-d9f3cbeb073a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.188218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.188834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.189115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.189386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.190025 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:10.190327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.191991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.204117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 625/1252] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:10.205705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.215781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.216399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.216681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.216793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1cbffc-18dd-406b-8bf7-e63b498d902a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.216955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.217590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.219788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.222484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.223074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.223359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.223634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.224272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:10.224573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.226684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.229509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 628/1253] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.230372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.240165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 626/1254] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:10.240608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df919cda-8304-4acb-a5cb-ade8dd59903e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.241452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.245272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.245868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.246151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.246424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.247263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.248975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.253172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ac1fe1-7580-4a6e-b139-c34096e4bc90 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.258215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.258832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.259536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.259816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.260438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.262774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.276523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 629/1255] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.277341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.288602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 627/1256] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 287 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.289405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.299421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da1438d-392c-46e3-9cc7-f55bd1609317 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.350412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983d1ba6-f4bf-4e3b-9ca6-6109e947d87f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.355367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.355574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.356004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.356180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.356305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.356461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.356579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.356745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.357168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.357342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.359115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.359567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.375537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.381983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 628/1257] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.382925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.387585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 630/1258] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/projects => generated 365 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.388501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.397374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a251942b-965a-45f3-8c63-60f6f9856bef admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.428698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.453411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b9fd98-8148-4f67-838a-297fc31e040b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.456099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.456653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.456934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.457206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.457860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.458014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.458645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.458923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.459191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.459528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.459807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.461711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.470684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 629/1259] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.471985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.483091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d2ebb1-65ba-4c37-a5b9-17d561fd1f03 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.488933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.489091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 631/1260] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/projects => generated 373 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.489695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.489974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.490076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.490302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] PATH_INFO: `/v3/projects/49a39f93e67c4d3f9529bdef5bb66b4c/users/006623bab8f04f448edf0d43397192d8/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.506745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a39f93e67c4d3f9529bdef5bb66b4c, user_id=006623bab8f04f448edf0d43397192d8, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.509510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.522674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 630/1261] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:10 2022] PUT /identity/v3/projects/49a39f93e67c4d3f9529bdef5bb66b4c/users/006623bab8f04f448edf0d43397192d8/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:10.523565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.556146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28059e3b-0b9c-40b5-b5a7-3315541ec6c8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.585819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378a0d1-a30e-4ef0-9b91-bbe4e62c31b9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.590762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.591418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.591720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.591996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.592676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.595041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.612258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.613037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.613346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.613646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.614323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.616272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.620362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 631/1262] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.620362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.630101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fd9a6c-2ff0-4113-8add-0054de7fde68 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.634775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.635362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.635636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.635904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.636553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.638130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.644276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 632/1263] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/projects => generated 385 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.644967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.653210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 632/1264] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.654058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.664007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debd94d7-1256-4ad6-b84d-4f522e2965e8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.700344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.712371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ec3f24-bea0-4611-b8ab-bf0423e20ed6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.714068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:10.717855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.719069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.719376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.719706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.720381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.722264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.729332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.729992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.730276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.730652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.731272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.732876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 633/1265] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.733215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.733742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.743573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cd5757-1131-4857-a1a8-c87828f6d30c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.748909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.750239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.750603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.750893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] PATH_INFO: `/v3/projects/4804846d26ce49df866f808594564022/users/50a2bbb6bef043ba906d9e0775e9c285/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.758907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 633/1266] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.758907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.767399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4804846d26ce49df866f808594564022, user_id=50a2bbb6bef043ba906d9e0775e9c285, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.768174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4e392-64f8-4010-ab8c-b81557d04d6c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.770098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9147b2b0-9eee-47e7-9fb7-af591bfdfa13 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.772652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.774484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.774484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.774484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.774484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.776286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.785085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 634/1267] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:10 2022] PUT /identity/v3/projects/4804846d26ce49df866f808594564022/users/50a2bbb6bef043ba906d9e0775e9c285/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:10.788738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.788738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.788738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.801348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 634/1268] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.802288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.807195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:10.808402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:10.812839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a04e3c5-a7ca-4571-af19-1e83f2bfa3b2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.868700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.869397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.869574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.869843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.870499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.872016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 635/1269] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:10.872130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.875445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.884339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 635/1270] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.884595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c1c68-fd5c-4ec5-920a-09e3e614603a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.888155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.889186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.889778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.890099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.890548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.891180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.893070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.898323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a84fae-1f31-4e00-ba5a-f8ca91fb9e75 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.902967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.903538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.903816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.904087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.904701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.907410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.917448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 636/1271] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.918246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.918651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 636/1272] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:10 2022] POST /identity/v3/users => generated 293 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:10.919541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.930291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0aed4a-4947-43fc-a6ed-ff89ebd6d87c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.930752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4f3873-ab2f-49b2-af95-2eb0fbff094d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.935296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.935899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.936019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.936185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.936461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.936571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.936759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.937591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.937917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.938256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.939604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.939896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.950646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 637/1273] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.951452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.957764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 637/1274] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.957764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.962047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082176dc-db97-4d35-8218-8f462631b336 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.965084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7300db0e-417b-496a-9ac4-e59e735d47bb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:10.966792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.967383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.967664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.967940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.969352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.970490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:10.971076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.971176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:10.971360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:10.971637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:10.972469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:10.974032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:10.981044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 638/1275] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.981907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.987179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 638/1276] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:10.987973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:10.997816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577cecef-fb1d-4075-b1b3-82190505765d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.003277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.003860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.004143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.004415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.005040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.006659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.017076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 639/1277] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:11.017845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.029495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6f5700-c1d5-4d85-a700-efb7ed9ceb89 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.034115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.034788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.035071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.035464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] PATH_INFO: `/v3/projects/8e99dc2520974d90b2f324fc84839685/users/38099f801c4d472598c6fea92caddaf9/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.037872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af723c7-2473-4829-8a9b-4631afa2dacb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.041493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.042084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.042362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.042661 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.049860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e99dc2520974d90b2f324fc84839685, user_id=38099f801c4d472598c6fea92caddaf9, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.052559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.053011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.054472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.065965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 639/1278] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:11.066828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 640/1279] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/8e99dc2520974d90b2f324fc84839685/users/38099f801c4d472598c6fea92caddaf9/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.066957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.067778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.076989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b245834-6f96-4834-b052-709779c75369 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.107737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.128979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.145291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c59fff-e568-4e9a-bdca-572085d3ed0c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.147299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.147932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.148239 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.148518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.149221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.150093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.150812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.150940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.151152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.151417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] PATH_INFO: `/v3/projects/d7ac3d840c9f41ea842b8f4935770f8b/users/2335878ea1d045858b158f6b4175e896/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.163360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 640/1280] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:11.167637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.171162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ac3d840c9f41ea842b8f4935770f8b, user_id=2335878ea1d045858b158f6b4175e896, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.175130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.179405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed3f07-fa91-43a7-9f25-6ec7f38647ce admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.185110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.185960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.186493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.186783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.188076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.189722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.192300 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 641/1281] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/d7ac3d840c9f41ea842b8f4935770f8b/users/2335878ea1d045858b158f6b4175e896/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.193083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.209375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 641/1282] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:11.209375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.218621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9444c8ff-e1d0-4a7f-985c-b5941f173066 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.277630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c20d839-55c7-4898-b480-864cb861e113 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.286696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.287377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.287682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.287959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] PATH_INFO: `/v3/projects/4804846d26ce49df866f808594564022/users/50a2bbb6bef043ba906d9e0775e9c285/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.289249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.289869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.290149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.290254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4804846d26ce49df866f808594564022, user_id=50a2bbb6bef043ba906d9e0775e9c285, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.290435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.291339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.293134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.293241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b7b01-60e9-41b6-9202-a87fdf4642c6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.305171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 642/1283] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:11.305882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.307586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 642/1284] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/4804846d26ce49df866f808594564022/users/50a2bbb6bef043ba906d9e0775e9c285/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.309321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.309665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.310023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.325489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9d96-1e1d-4b2e-ab31-0e3989941a3d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.341148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] MFA Rules not processed for user `38099f801c4d472598c6fea92caddaf9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:11.342760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:11.394795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.395449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.395731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.396069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] PATH_INFO: `/v3/projects/d296608fc23c4667b5173c0f3fe1697e/users/5495217023fc44eab2fa73552db6b3f2/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.415940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d296608fc23c4667b5173c0f3fe1697e, user_id=5495217023fc44eab2fa73552db6b3f2, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.418665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93d31f4-d1fb-483e-acb3-298fe2e1803b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.434799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 643/1285] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:11.436261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.446021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 643/1286] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/d296608fc23c4667b5173c0f3fe1697e/users/5495217023fc44eab2fa73552db6b3f2/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.446021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.446021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.446021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.454250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b22f0f-526e-40d7-8708-c90935f8812e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.464652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:11.466206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:11.498083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.515773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.523217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.523444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.523701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.523975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.524816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.527336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.539367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 644/1287] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:11.540223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.544043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 644/1288] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:11.544043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.550802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5fb617-064a-4900-8d9b-7087762053c2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.555997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.556664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.556943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.557217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] PATH_INFO: `/v3/projects/b19eadcff3d740c7bd8cc0952474a765/users/a6a2d2c330b74681baff40009faa119c/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.558608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5697b533-e957-40e2-8254-345e7e83fba9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.564733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.565432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.565713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.565989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] PATH_INFO: `/v3/projects/f7620aafbe254b9aa92329275c1b6738/users/284489374b204776af052704ac522bcf/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.573990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b19eadcff3d740c7bd8cc0952474a765, user_id=a6a2d2c330b74681baff40009faa119c, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.576691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edcd413-bb75-4ce5-9e72-2de6ed889bba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.581386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7620aafbe254b9aa92329275c1b6738, user_id=284489374b204776af052704ac522bcf, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.584217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.592944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 645/1289] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/b19eadcff3d740c7bd8cc0952474a765/users/a6a2d2c330b74681baff40009faa119c/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.595161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.595451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.595727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.608724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 645/1290] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/f7620aafbe254b9aa92329275c1b6738/users/284489374b204776af052704ac522bcf/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.609395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.635187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] MFA Rules not processed for user `50a2bbb6bef043ba906d9e0775e9c285`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:11.637428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:11.678592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12213ca-2c10-4453-9eb5-7a8218d4b6e4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.693273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.694068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.694348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.694676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.695304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.697200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbcad6-7be9-4bdf-bfc9-adc58d989a0c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.728453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 646/1291] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:11.729735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.730028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.730305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.734195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 646/1292] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:11.734195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.751676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11fc898-097b-40a2-a29b-0812fd7691a4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.754050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:11.755908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:11.839677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.840403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.840944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.841256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] PATH_INFO: `/v3/projects/fa4d62bf272e4ce69022518ff362813f/users/c9dba6c5f7e9472faa5aa963758d78fe/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.845520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 647/1293] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:11.845520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:11.870430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa4d62bf272e4ce69022518ff362813f, user_id=c9dba6c5f7e9472faa5aa963758d78fe, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:11.874683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcbf467-8903-4647-9e3d-a9d7b22b1ec2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:11.904523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 647/1294] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:11 2022] PUT /identity/v3/projects/fa4d62bf272e4ce69022518ff362813f/users/c9dba6c5f7e9472faa5aa963758d78fe/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:11.905621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9a96-9686-40ea-85b4-ee2f91bb8f27 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.906465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9a96-9686-40ea-85b4-ee2f91bb8f27 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.906748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fa9a96-9686-40ea-85b4-ee2f91bb8f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:11.921271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32037f86-3254-4e9e-94a2-e7ce7801b486 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:11.931602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.947255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-11fa9a96-9686-40ea-85b4-ee2f91bb8f27 None None] MFA Rules not processed for user `a6a2d2c330b74681baff40009faa119c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:11.950312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-11fa9a96-9686-40ea-85b4-ee2f91bb8f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:11.957155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:11.988403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:11.989194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:11.989483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:11.989883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.047420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 648/1295] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:12.049117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.049414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.049739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.068572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.070784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.084816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 648/1296] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:12.085417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] MFA Rules not processed for user `284489374b204776af052704ac522bcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:12.086009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.087147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:12.098217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f86cf68-606f-414a-b41f-7bd6b948e043 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.103896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.104639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.105023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.105449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.106184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.109190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.140745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 649/1297] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/users => generated 293 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:12.140745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.165876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 649/1298] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:12.165876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.177004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be3995d-5a9a-442d-97f8-f0de0d634be6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.181381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.181922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.182208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.182570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.194369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.195097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0edb641-82f6-48b6-ab35-8d6a0acb753c service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.206110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 650/1299] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:12.207416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.207703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.207980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.228579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:12.229806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:12.244991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 651/1300] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:12.245986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.258357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d5593-0dac-4a85-8b7c-0daf2ae771be None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.265656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.267728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.267728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.267728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.268035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.270218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.307558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 652/1301] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/projects => generated 363 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:12.309021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.318884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f105b744-495a-4d2f-abcb-b0a4e709d2d3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.380757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:12.398102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90abebf-c249-463c-a307-1c07f4cfda37 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.405461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.407161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.407721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.408113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.408973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.409188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.409930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.410324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.410711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.411030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d48ea-358f-43f6-b535-1a2c8f15644a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.426985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 653/1302] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:12.428894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.429322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.429607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.431283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 654/1303] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:12.431773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.434407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:12.437425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:12.521911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.524262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.525187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e4d08e-5398-456c-9615-dc1331c6febf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.530056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.532043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.533360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.533649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.540131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 650/1304] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:12.542619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.557734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457f37e-f0e8-45f8-8ddf-44981a98370a service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.565095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.566146 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.566597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.566711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.568243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.568981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.569763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.571148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50f3c79-9605-40b5-bfe5-c7184dda628c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.602479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 655/1305] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:12.602479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.624795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 651/1306] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/users => generated 288 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:12.626399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.626713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.626992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.659716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa2267b-bc84-4b91-8f5c-3f513d3dbf3e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.665506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.666201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.666543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.666852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] PATH_INFO: `/v3/projects/69cee9e1b58448b3aa1148879c2d06b0/users/d4a2790dc4bc4c0192b196f935f15615/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.700887 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] RBAC: Authorizing `identity:create_grant(project_id=69cee9e1b58448b3aa1148879c2d06b0, user_id=d4a2790dc4bc4c0192b196f935f15615, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.707739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:12.708763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.710591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:12.749484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 656/1307] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:12 2022] PUT /identity/v3/projects/69cee9e1b58448b3aa1148879c2d06b0/users/d4a2790dc4bc4c0192b196f935f15615/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:12.750526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.841583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 652/1308] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:11:12 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:12.842527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:12.853863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91b602-5884-4ad0-98ff-60fbb54046bc None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.941622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df35b350-a7cd-4af6-a840-0030b189f03e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:12.954406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.956172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.956550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.957675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.958679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.960079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.979492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 657/1309] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:12.981091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:12.981477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:12.981764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:12.982044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:12.984506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:12.986363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:12.989485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.011164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 653/1310] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:13.011164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.029187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:13.160696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3797b-2925-4d70-b5e5-b83fd090df52 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.160696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.161505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99c92e-8ad7-41c1-be50-8bdd7111ca68 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.161973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.161973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.161973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.169291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.169941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.170219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.170562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.200286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:13.289095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.291361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.317467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 658/1311] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:13.319011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.319712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.321519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.340855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 654/1312] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:13 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:13.342420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.345241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7939c2-d1d5-4e43-b098-b8c4014a2df5 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.355401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bf7182-79ec-4495-9b6b-51b63cc2a0ad service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.360466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.361353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.361655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.362108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.362269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.362713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.362986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] PATH_INFO: `/v3/projects/064bf192c42b4d82a89e9591788b72ae/users/8da7fe3787f541219a1ad351c4138460/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.364060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] PATH_INFO: `/v3/projects/69cee9e1b58448b3aa1148879c2d06b0/users/d4a2790dc4bc4c0192b196f935f15615/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.366193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] RBAC: Authorizing `identity:create_grant(project_id=69cee9e1b58448b3aa1148879c2d06b0, user_id=d4a2790dc4bc4c0192b196f935f15615, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.369470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0906f52-4774-4798-a75d-09b3e2e8785b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.395357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 655/1313] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:13 2022] PUT /identity/v3/projects/69cee9e1b58448b3aa1148879c2d06b0/users/d4a2790dc4bc4c0192b196f935f15615/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:13.397493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.397911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.398355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.402818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=064bf192c42b4d82a89e9591788b72ae, user_id=8da7fe3787f541219a1ad351c4138460, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.407975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.434703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 659/1314] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:13 2022] PUT /identity/v3/projects/064bf192c42b4d82a89e9591788b72ae/users/8da7fe3787f541219a1ad351c4138460/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:13.439064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] MFA Rules not processed for user `d4a2790dc4bc4c0192b196f935f15615`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:13.439565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.441258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:13.579553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3716734-f657-4775-bb8d-b25d072225fc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.594102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.594763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.595136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.595300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.595950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.597856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.604306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 656/1315] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:13.621980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.628954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 660/1316] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:13 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:13.633670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:13.659416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:13.660209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe89621-f70c-489e-8cb5-e95518319fa0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.678098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.680893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.681161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.681529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] PATH_INFO: `/v3/projects/064bf192c42b4d82a89e9591788b72ae/users/8da7fe3787f541219a1ad351c4138460/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.686248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=064bf192c42b4d82a89e9591788b72ae, user_id=8da7fe3787f541219a1ad351c4138460, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:13.694247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d700337-555f-4e61-8071-9bb3d6d37e18 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:13.734384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 661/1317] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:13 2022] PUT /identity/v3/projects/064bf192c42b4d82a89e9591788b72ae/users/8da7fe3787f541219a1ad351c4138460/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:13.735952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3c1e-4759-4802-95a9-81b6adba7049 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.736270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3c1e-4759-4802-95a9-81b6adba7049 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.736829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ebc3c1e-4759-4802-95a9-81b6adba7049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:13.769592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2014b-61fb-4329-b0be-60bc756d7ebf None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:13.841456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3ebc3c1e-4759-4802-95a9-81b6adba7049 None None] MFA Rules not processed for user `8da7fe3787f541219a1ad351c4138460`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:13.846747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3ebc3c1e-4759-4802-95a9-81b6adba7049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:13.944658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:13.945299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:13.945583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:13.945861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.016286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 662/1318] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:13 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:14.019026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385c3e-80b5-4dc8-92a0-976eba4ca4a6 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:14.019528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385c3e-80b5-4dc8-92a0-976eba4ca4a6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:14.020028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2385c3e-80b5-4dc8-92a0-976eba4ca4a6 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.022006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 663/1319] 10.222.0.23 () {54 vars in 912 bytes} [Sun Oct 23 02:11:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:14.033584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:14.034232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:14.034902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.055294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:14.074900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:14.076587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:14.097142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:14.104642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:14.129772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 657/1320] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:13 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:14.131649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:14.153198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847649c8-26ff-476e-a897-02257d1f4775 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:14.161257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:14.163339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:14.164623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:14.165015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.189349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:14.193096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:14.220962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 658/1321] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:14 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:14.300390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 664/1322] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:11:14 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:14.337820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:14.352287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743db8a-eaf3-4b36-b385-ee478cfa3613 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:14.360115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:14.362004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:14.362402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:14.362962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.393753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:14.395590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ea973c-f64a-476c-9422-3c8d012c1684 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:14.426344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 659/1323] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:14.502930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:14.514469 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f24e767-e280-456a-bff1-efdb13a49c54 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:14.525298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:14.526113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:14.526519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:14.526903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:14.663020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:14.665681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5e03f-b6d9-4c72-b9f4-16035c685f9c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:14.703647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 665/1324] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:15.909999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97faa-541a-4252-887a-6c9117905094 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:15.910714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97faa-541a-4252-887a-6c9117905094 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:15.910714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f97faa-541a-4252-887a-6c9117905094 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:15.912270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 660/1325] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:15.932591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:15.933165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:15.933165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:15.971493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:15.980056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:16.008434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75010-8809-4138-bb18-9be671eecb02 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.008963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75010-8809-4138-bb18-9be671eecb02 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.009412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f75010-8809-4138-bb18-9be671eecb02 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.011406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 661/1326] 10.222.0.23 () {54 vars in 912 bytes} [Sun Oct 23 02:11:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Oct 23 02:11:16.015485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 666/1327] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:11:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:16.017511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.017982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.018429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.037386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:16.066921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:16.071873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:16.107789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 662/1328] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:11:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:16.119241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:16.136735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97c697a-1817-48c3-b355-39869af758a8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:16.144333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:16.144960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.145241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.145715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.177867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:16.178960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0347261-c638-4530-a387-a83d7fb7f15d service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:16.198918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 663/1329] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:16.286173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d32b711-f614-4630-825d-8c7448d8dd89 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:16.299218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:16.299873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.300156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.304559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] PATH_INFO: `/v3/users/38099f801c4d472598c6fea92caddaf9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.306709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38099f801c4d472598c6fea92caddaf9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:16.309844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:16.409264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:16.444068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38099f801c4d472598c6fea92caddaf9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:16.458217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38099f801c4d472598c6fea92caddaf9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:16.470257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38099f801c4d472598c6fea92caddaf9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:16.471835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38099f801c4d472598c6fea92caddaf9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:16.472742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38099f801c4d472598c6fea92caddaf9'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:16.498104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 667/1330] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:16 2022] DELETE /identity/v3/users/38099f801c4d472598c6fea92caddaf9 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:16.502993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.503796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.503796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.538348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:16.541081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:16.583254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:16.625965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 664/1331] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:16.666948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:16.669280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:16.783673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbdfc20-5173-462c-af92-686a8fa10692 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:16.795038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:16.796695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:16.797297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:16.797790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:16.826042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:16.832691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:16.859791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 668/1332] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:11:19.392609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:19.395921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:19.416876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b643361-d43e-4897-8ee0-b4e64630a17f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:19.429463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:19.430933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:19.431296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:19.433271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] PATH_INFO: `/v3/projects/8e99dc2520974d90b2f324fc84839685` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:19.435065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e99dc2520974d90b2f324fc84839685)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:19.456089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:19.507969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:19.507969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:19.534270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1f5ab-c9fa-4a4b-8dda-effd88740490 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:19.564705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:19.571602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:19.571885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:19.572159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] PATH_INFO: `/v3/users/284489374b204776af052704ac522bcf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:19.581776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] RBAC: Authorizing `identity:delete_user(user_id=284489374b204776af052704ac522bcf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:19.587345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:19.629610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e99dc2520974d90b2f324fc84839685', 'request_id': 'req-0355f964-c093-4f88-9a00-707d4c3b2759'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.651165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e99dc2520974d90b2f324fc84839685', 'request_id': 'req-0355f964-c093-4f88-9a00-707d4c3b2759'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.672038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e99dc2520974d90b2f324fc84839685', 'request_id': 'req-0355f964-c093-4f88-9a00-707d4c3b2759'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.708113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] The token cache is being invalidate because project 8e99dc2520974d90b2f324fc84839685 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:19.708113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.708113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.724754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 665/1333] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:19 2022] DELETE /identity/v3/projects/8e99dc2520974d90b2f324fc84839685 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:19.725704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:19.728580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:19.747235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '284489374b204776af052704ac522bcf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.781576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '284489374b204776af052704ac522bcf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.809194 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '284489374b204776af052704ac522bcf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.810156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '284489374b204776af052704ac522bcf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.810969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-29038a36-cb7b-4988-900e-99a394e0250f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '284489374b204776af052704ac522bcf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:19.872478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 669/1334] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:19 2022] DELETE /identity/v3/users/284489374b204776af052704ac522bcf => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:19.874736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:19.875027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:19.875305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:19.903189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0355f964-c093-4f88-9a00-707d4c3b2759 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:19.927941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:19.930732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:19.964453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:20.076764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 670/1335] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:20.094206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:20.096657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:20.096657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:20.097948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] PATH_INFO: `/v3/users/d4a2790dc4bc4c0192b196f935f15615` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:20.099491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4a2790dc4bc4c0192b196f935f15615)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:20.102837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: WARNING py.warnings [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:11:20.102837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: warnings.warn(msg) Oct 23 02:11:20.102837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: Oct 23 02:11:20.104059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:20.127750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:20.133742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:20.136277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:20.307450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4a2790dc4bc4c0192b196f935f15615'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:20.331710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd7799c-d0de-480c-b845-1035e15ebcf6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:20.335920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4a2790dc4bc4c0192b196f935f15615'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:20.338296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4a2790dc4bc4c0192b196f935f15615'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:20.350741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:20.351776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:20.351924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:20.352237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4a2790dc4bc4c0192b196f935f15615'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:20.352408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:20.353822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c16aa98-53b6-4f6c-a59e-04e4242af7ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4a2790dc4bc4c0192b196f935f15615'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:20.359950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:20.408516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:20.410224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:20.419342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 666/1336] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:19 2022] DELETE /identity/v3/users/d4a2790dc4bc4c0192b196f935f15615 => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:20.448660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 671/1337] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:11:20.688300 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:20.688839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:20.692147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:20.734401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] MFA Rules not processed for user `c9dba6c5f7e9472faa5aa963758d78fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:20.737457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:20.915088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 667/1338] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:20.937606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:20.939663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:21.183313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0a670f-798b-4075-a63e-f433ce0948cb service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:21.355782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:21.389783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:21.396442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:21.396773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:21.397006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:21.409019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:21.411148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2654f9aa-d3ca-4386-9108-76d6d2fe50b9 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:21.447915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 672/1339] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:22.609986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:22.611505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:22.770654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76dd59-28bc-4aca-8cb8-1683f1da41b7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:22.776758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:22.777657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:22.780223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:22.781349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] PATH_INFO: `/v3/projects/69cee9e1b58448b3aa1148879c2d06b0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:22.782857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69cee9e1b58448b3aa1148879c2d06b0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:22.786253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:22.809269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:22.869388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69cee9e1b58448b3aa1148879c2d06b0', 'request_id': 'req-e1433dcc-5eb2-47ef-aaad-be362c6df621'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:22.878262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69cee9e1b58448b3aa1148879c2d06b0', 'request_id': 'req-e1433dcc-5eb2-47ef-aaad-be362c6df621'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:22.886271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69cee9e1b58448b3aa1148879c2d06b0', 'request_id': 'req-e1433dcc-5eb2-47ef-aaad-be362c6df621'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:22.979374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] The token cache is being invalidate because project 69cee9e1b58448b3aa1148879c2d06b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:22.979887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:22.983064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:22.992079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 668/1340] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:22 2022] DELETE /identity/v3/projects/69cee9e1b58448b3aa1148879c2d06b0 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:22.996640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946448-e986-4d9a-ac38-d3e4c8933e9a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.006691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946448-e986-4d9a-ac38-d3e4c8933e9a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.007373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35946448-e986-4d9a-ac38-d3e4c8933e9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.080275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:23.081936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.087078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-35946448-e986-4d9a-ac38-d3e4c8933e9a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:23.088483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-35946448-e986-4d9a-ac38-d3e4c8933e9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.227955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1433dcc-5eb2-47ef-aaad-be362c6df621 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:23.251307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:23.252117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.252451 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.252737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] PATH_INFO: `/v3/projects/f7620aafbe254b9aa92329275c1b6738` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.258871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7620aafbe254b9aa92329275c1b6738)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:23.260959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:23.306816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 673/1341] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:23.308936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.309232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.309512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.371352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:23.372658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.419468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 674/1342] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:11:23.425372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:23.445074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7620aafbe254b9aa92329275c1b6738', 'request_id': 'req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:23.458960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7620aafbe254b9aa92329275c1b6738', 'request_id': 'req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:23.470312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7620aafbe254b9aa92329275c1b6738', 'request_id': 'req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:23.480175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341107c7-9aa9-4039-b40f-71fa8c7d43ef None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:23.494563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] The token cache is being invalidate because project f7620aafbe254b9aa92329275c1b6738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:23.494906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:23.495765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a9b1620f-48a0-4c5d-a822-768e7d9b4ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:23.502212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 669/1343] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:23 2022] DELETE /identity/v3/projects/f7620aafbe254b9aa92329275c1b6738 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:23.503450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0128-505c-4606-a18f-5b2d13e46aba None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.503735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0128-505c-4606-a18f-5b2d13e46aba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.504030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dc0128-505c-4606-a18f-5b2d13e46aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.530045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-01dc0128-505c-4606-a18f-5b2d13e46aba None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:23.532347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-01dc0128-505c-4606-a18f-5b2d13e46aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.591822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:23.615349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:23.616703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.616983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.617418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.620610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:23.621376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:23.626677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:23.659389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 675/1344] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:23.667310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:23.670259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.677312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 670/1345] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:23.695423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.696986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.697288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.726960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e129ba17-3d32-4715-94f7-8fff3ba0d73c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:23.740348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:23.741352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.741352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.741556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.742044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:23.744182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:23.753381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:23.755732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:23.780617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 671/1346] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:23.788857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:23.808784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 676/1347] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:23 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:23.810400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:23.815987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b1049-9cfd-4eb0-ac2b-c5c977c16d6a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:23.980530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:23.997327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:23.997327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:23.997327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:23.997327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:23.998361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:23.998361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:23.998361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.005852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25d86a7-cc45-4335-a5e2-f71f6d51f63c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.018017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.018706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.019031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.019302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.019904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.023634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.025290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 672/1348] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:24.026050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.052308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e7279c-223c-4a04-99f4-976c9d2cde49 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.064812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.066461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.066579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 677/1349] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:23 2022] POST /identity/v3/users => generated 283 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:24.070495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.070723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.070874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.071613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.073879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.092692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fae425-94ec-43a7-8dbc-5b0cdea5fa9e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.103197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.103856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.104169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.104586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.105211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.115156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.121017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 673/1350] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:24 2022] POST /identity/v3/projects => generated 383 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:24.122824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.141825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 678/1351] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:24.145510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.163345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62948a-41a1-4ec3-b625-d7d5097a4602 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.238887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd0c55f-7742-4fb4-8c60-0008624a220e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.251584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.252422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.252980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.255356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.256794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.264299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.316326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 674/1352] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:24 2022] POST /identity/v3/users => generated 292 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:24.317371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.318166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.321701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.322850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.323245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.324006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.330883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.344872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c225acd-60c2-4add-8410-c2c74f434003 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.358379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 679/1353] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:24.359561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.360295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.360951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.361265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.361585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.362333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.364093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.380842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ca613d-209d-4fd6-aaa9-2677048a1428 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.386796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.387596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.387982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.388587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] PATH_INFO: `/v3/projects/fd203962e98e40dea828d8223b1d5a58/users/a047b5a1a3894dc299b8663f2cfba22c/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.390660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 675/1354] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:24.394612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.412159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fabfa7a-0ceb-44a8-bd78-02ddc033ba3a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.422184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd203962e98e40dea828d8223b1d5a58, user_id=a047b5a1a3894dc299b8663f2cfba22c, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.424724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.425360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.425645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.425922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.426627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.427332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593ee2e8-9988-4870-8429-07724bab02a4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.428551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.455396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 680/1355] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:24 2022] PUT /identity/v3/projects/fd203962e98e40dea828d8223b1d5a58/users/a047b5a1a3894dc299b8663f2cfba22c/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:24.466193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a35e9-5563-487b-b806-68b7cb074150 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.469719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a35e9-5563-487b-b806-68b7cb074150 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.470167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60a35e9-5563-487b-b806-68b7cb074150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.480696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 676/1356] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:24.481636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:24.497444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023b5dda-23c6-4999-b760-df51b56b2737 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:24.506519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:24.547225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c60a35e9-5563-487b-b806-68b7cb074150 None None] MFA Rules not processed for user `a047b5a1a3894dc299b8663f2cfba22c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:24.551071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c60a35e9-5563-487b-b806-68b7cb074150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:24.616571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:24.617296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.617578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.618711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] PATH_INFO: `/v3/projects/79275f37a32140718ec85a2551b4180f/users/d39e30066a8d44458a3cc3d6e0c63883/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.662456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] RBAC: Authorizing `identity:create_grant(project_id=79275f37a32140718ec85a2551b4180f, user_id=d39e30066a8d44458a3cc3d6e0c63883, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:24.665680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f65b8cd-ac58-447b-8b2f-6c51c91d587c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:24.771804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 677/1357] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:24 2022] PUT /identity/v3/projects/79275f37a32140718ec85a2551b4180f/users/d39e30066a8d44458a3cc3d6e0c63883/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:24.773387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.773688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.773970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.810082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 681/1358] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:24 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:24.815749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:24.816242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:24.822170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:24.823454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] MFA Rules not processed for user `c9dba6c5f7e9472faa5aa963758d78fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:24.829380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:24.900899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] MFA Rules not processed for user `d39e30066a8d44458a3cc3d6e0c63883`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:24.902451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:24.947459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:25.004200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 678/1359] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:25.011212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.074675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 682/1360] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:24 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:25.076024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.263836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a91cb97-c3e8-47f3-b750-f4d5f324594d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.278429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.279892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.284281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.285873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.286793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.292337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.322286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1978eec-5600-41a9-a690-c4645f50d4c5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.340198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.340837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.344561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.346539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.347245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.351909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.414749 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 679/1361] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:25 2022] POST /identity/v3/projects => generated 363 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:25.419732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.447374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 683/1362] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:25 2022] POST /identity/v3/projects => generated 383 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:25.455351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.485938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:25.575845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1ffa8a-3728-4153-8f3d-ef7808d06747 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.579550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f31424-884b-4a9e-b29e-10f1be25dc76 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.588037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.590294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.590757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.591193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.591477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.592507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.596005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.596298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.596583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.597593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.600189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.600928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.658485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 680/1363] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:25 2022] POST /identity/v3/users => generated 288 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:25.659486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.669370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 684/1364] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:25 2022] POST /identity/v3/users => generated 298 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:25.673775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.696515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3fb646-1c50-4bd3-9998-aa387e3ff56a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.701872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.702582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.702868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.703149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.703500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d05263-8d84-455d-902d-1f3aa07f5863 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.703801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.705489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.714546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.715344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.716046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.716046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.717861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.720734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.744099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 681/1365] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:25.745101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.753141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 685/1366] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:25.759279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.769614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e855fd4-4b1e-4cf4-9192-6d4a2cd4bfed admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.781204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.784074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.784388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.784674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] PATH_INFO: `/v3/projects/1d0ae24e18ee431ca4fed3a21593d648/users/8b93ac8ce86d41e989c3c42383beaea1/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.787348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0e20de-c4ba-4b0d-82f4-f9b94b425d37 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:25.807053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:25.808160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:25.808518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:25.808868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] PATH_INFO: `/v3/projects/6500f18ae566427bbdd5e65fcc390fda/users/6d3795c179574162a258093e81db6dc5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:25.819284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0ae24e18ee431ca4fed3a21593d648, user_id=8b93ac8ce86d41e989c3c42383beaea1, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.822020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.854126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 682/1367] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:25 2022] PUT /identity/v3/projects/1d0ae24e18ee431ca4fed3a21593d648/users/8b93ac8ce86d41e989c3c42383beaea1/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:25.860648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.872582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6500f18ae566427bbdd5e65fcc390fda, user_id=6d3795c179574162a258093e81db6dc5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:25.882259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:25.912226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 686/1368] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:25 2022] PUT /identity/v3/projects/6500f18ae566427bbdd5e65fcc390fda/users/6d3795c179574162a258093e81db6dc5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:25.915987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:25.926497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:26.033567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1918943-1ad1-4669-814e-3d595690fbbb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:26.126903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1921ee4b-abc5-4f09-b47d-64c5aecb6731 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:26.138099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:26.139286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.139893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.140327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.141068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:26.142908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:26.173941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 687/1369] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:26.174954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:26.206998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4c3926-c5e4-4e02-9867-ee89ccc0f1d9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:26.222873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:26.223628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.223896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.224176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] PATH_INFO: `/v3/projects/6500f18ae566427bbdd5e65fcc390fda/users/6d3795c179574162a258093e81db6dc5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.226189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:26.226738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.227131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.227541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.228226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:26.229421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6500f18ae566427bbdd5e65fcc390fda, user_id=6d3795c179574162a258093e81db6dc5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:26.230010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55792c8-00cb-4bb2-ab08-c6028857cfb4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:26.233090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:26.256911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 688/1370] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:26 2022] PUT /identity/v3/projects/6500f18ae566427bbdd5e65fcc390fda/users/6d3795c179574162a258093e81db6dc5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:26.263305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 683/1371] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:26.270222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:26.272287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021633-b7d2-493c-acb1-25e0b9b12e38 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.272459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021633-b7d2-493c-acb1-25e0b9b12e38 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.272745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3021633-b7d2-493c-acb1-25e0b9b12e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.408475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d3021633-b7d2-493c-acb1-25e0b9b12e38 None None] MFA Rules not processed for user `6d3795c179574162a258093e81db6dc5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:26.411918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d3021633-b7d2-493c-acb1-25e0b9b12e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:26.453740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:26.524292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612e5065-0c20-4fc9-93fd-77cec06f12dc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:26.555936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:26.556679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.556950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.557280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] PATH_INFO: `/v3/projects/1d0ae24e18ee431ca4fed3a21593d648/users/8b93ac8ce86d41e989c3c42383beaea1/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.561979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d0ae24e18ee431ca4fed3a21593d648, user_id=8b93ac8ce86d41e989c3c42383beaea1, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:26.564716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9552a945-6b5d-4fa1-b96d-f77997a30f58 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:26.594238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 689/1372] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:26 2022] PUT /identity/v3/projects/1d0ae24e18ee431ca4fed3a21593d648/users/8b93ac8ce86d41e989c3c42383beaea1/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:26.597027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.597312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.597586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.636234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 684/1373] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 365 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:26.638004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:26.638295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:26.649762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:26.681041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] MFA Rules not processed for user `006623bab8f04f448edf0d43397192d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:26.683100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:26.726350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] MFA Rules not processed for user `8b93ac8ce86d41e989c3c42383beaea1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:26.727454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:26.861131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 690/1374] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:26.869417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:26.869417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:26.898949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:26.921848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 685/1375] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:26.925747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:26.934305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:27.029857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454e8cb-7013-4fac-95e0-f58bb4eede99 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.034800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.035495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.035784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.036050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.063643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd75aaf-86a7-470a-8070-e34faf16dd05 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.074942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.076895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.077191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.077590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.094686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:27.096315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:27.131667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 686/1376] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:27.133937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:27.136583 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:27.170844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5504b6f-1af0-4248-94a9-ed17d82acf6e service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.178567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.180627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.181974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.183995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.212329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:27.214223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:27.219367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195f19c2-97bc-48e3-bd4a-27d5012651e3 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:27.227885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94699-546c-44ef-b0b6-3d92b4559b7d service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:27.272002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 691/1377] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:26 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:27.275668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 687/1378] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:27 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:27.279798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a112b5-7622-4065-bc8a-e48215f4d937 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.280998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a112b5-7622-4065-bc8a-e48215f4d937 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.282149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a112b5-7622-4065-bc8a-e48215f4d937 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.296596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.297623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 692/1379] 10.222.0.23 () {54 vars in 912 bytes} [Sun Oct 23 02:11:27 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:27.308825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.309126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.332105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:27.450207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:27.451993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:27.498309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a112b5-7622-4065-bc8a-e48215f4d937 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:27.520031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a112b5-7622-4065-bc8a-e48215f4d937 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.532671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.533230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.533586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.533863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.608341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 688/1380] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:11:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:27.640971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:27.642615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:27.666715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab95cc1-37ca-464f-a08c-00ce88df8968 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.677172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:27.678672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:27.704461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 693/1381] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:11:27.718828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.722647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.722964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.723259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:27.771063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:27.773707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:27.807673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 689/1382] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:27.904796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:27.915060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d99dd49-d6d2-4f0a-be29-367fa2a7e43c service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:27.924281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:27.925017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:27.925382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:27.925685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:28.070737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:28.075454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:28.077227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:28.100431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 694/1383] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:29.292076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:29.294189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:29.578953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8be62d-6e99-4f2b-b6be-6916dcf71be7 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:29.600906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b13869-9003-4237-a980-fefa602164bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:29.601942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13869-9003-4237-a980-fefa602164bf service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:29.602507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13869-9003-4237-a980-fefa602164bf service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:29.603307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13869-9003-4237-a980-fefa602164bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:29.612632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-96b13869-9003-4237-a980-fefa602164bf service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:29.705954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b13869-9003-4237-a980-fefa602164bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:29.710262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b13869-9003-4237-a980-fefa602164bf service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:29.752029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 690/1384] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:31.102215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:31.290764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19140c34-482f-461d-a758-fa82730c2d85 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:31.304617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:31.306273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:31.310651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:31.311136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] PATH_INFO: `/v3/users/a047b5a1a3894dc299b8663f2cfba22c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:31.312857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a047b5a1a3894dc299b8663f2cfba22c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:31.317711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:31.454247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9414-d3c7-4558-a699-a61bcd97eac7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:31.454852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9414-d3c7-4558-a699-a61bcd97eac7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:31.454852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00c9414-d3c7-4558-a699-a61bcd97eac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:31.494358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a047b5a1a3894dc299b8663f2cfba22c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:31.524129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a047b5a1a3894dc299b8663f2cfba22c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:31.543895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a047b5a1a3894dc299b8663f2cfba22c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:31.546037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a047b5a1a3894dc299b8663f2cfba22c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:31.546819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-47410fb3-a94c-4615-9377-6605808c0525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a047b5a1a3894dc299b8663f2cfba22c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:31.553379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d00c9414-d3c7-4558-a699-a61bcd97eac7 None None] MFA Rules not processed for user `2335878ea1d045858b158f6b4175e896`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:31.554988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d00c9414-d3c7-4558-a699-a61bcd97eac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:31.592749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 695/1385] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:31 2022] DELETE /identity/v3/users/a047b5a1a3894dc299b8663f2cfba22c => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:31.598204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b69a5-ab2b-4382-8665-135490bd05f8 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:31.598518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b69a5-ab2b-4382-8665-135490bd05f8 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:31.600963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9b69a5-ab2b-4382-8665-135490bd05f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:31.691180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fc9b69a5-ab2b-4382-8665-135490bd05f8 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:31.697192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc9b69a5-ab2b-4382-8665-135490bd05f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:31.791427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 691/1386] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:11:31.793950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:31.794351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:31.794808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:31.812245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:31.850517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] MFA Rules not processed for user `2335878ea1d045858b158f6b4175e896`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:31.857112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.061873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 696/1387] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:32.067627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.068529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 692/1388] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:31 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:11:32.069612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:32.070917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.071750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.072579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.119581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f7c320-3a9a-49dc-9595-56c3c47db5fd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:32.150797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:32.151607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.151918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.152451 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] PATH_INFO: `/v3/users/d39e30066a8d44458a3cc3d6e0c63883` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.155136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=d39e30066a8d44458a3cc3d6e0c63883)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:32.158376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:32.170481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] MFA Rules not processed for user `5495217023fc44eab2fa73552db6b3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:32.174944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.181400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:32.308384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 697/1389] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:32.310767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:32.312540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.351465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd39e30066a8d44458a3cc3d6e0c63883'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.365429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd39e30066a8d44458a3cc3d6e0c63883'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.366633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd39e30066a8d44458a3cc3d6e0c63883'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.377021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd39e30066a8d44458a3cc3d6e0c63883'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.378218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd39e30066a8d44458a3cc3d6e0c63883'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.403900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 693/1390] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:32 2022] DELETE /identity/v3/users/d39e30066a8d44458a3cc3d6e0c63883 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:32.404313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:32.408271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.441275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18524b05-aeeb-4377-ad52-56dbc0f2f42a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:32.515763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d8748-83a3-450a-a1c1-31ad4b934aea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:32.525746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:32.526562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.526846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.527119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.529835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.550907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:32.551864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.552209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.552485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.554110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:32.555610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.627174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:32.630035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18158ebc-d539-432d-a084-420ec47f0273 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:32.651168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 694/1391] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:32 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:32.655353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd05e2-7123-4b59-b6f6-61b20bf90068 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.655779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd05e2-7123-4b59-b6f6-61b20bf90068 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.656177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcd05e2-7123-4b59-b6f6-61b20bf90068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.662705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:32.664756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5784a-8834-4df8-bb0c-3f4344131141 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:32.680249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 698/1392] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:32.681526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.681777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.682055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.701306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cdcd05e2-7123-4b59-b6f6-61b20bf90068 None None] MFA Rules not processed for user `5495217023fc44eab2fa73552db6b3f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:32.705133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cdcd05e2-7123-4b59-b6f6-61b20bf90068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.738581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:32.739663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.762103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 699/1393] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:32.763055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:32.764460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.785582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:32.789675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709dd714-0f29-46e6-b440-29affc19f902 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:32.804922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:32.805578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.805857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.806132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] PATH_INFO: `/v3/users/a6a2d2c330b74681baff40009faa119c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.807314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6a2d2c330b74681baff40009faa119c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:32.809242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:32.859288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 695/1394] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:32 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:32.862409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:32.862816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:32.863202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:32.911781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6a2d2c330b74681baff40009faa119c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.914398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] MFA Rules not processed for user `006623bab8f04f448edf0d43397192d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:32.915687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:32.932956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6a2d2c330b74681baff40009faa119c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.946947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6a2d2c330b74681baff40009faa119c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.948508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6a2d2c330b74681baff40009faa119c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.949021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6a2d2c330b74681baff40009faa119c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:32.974524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 700/1395] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:32 2022] DELETE /identity/v3/users/a6a2d2c330b74681baff40009faa119c => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:32.975584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:32.977534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:33.002894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 696/1396] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:33.003844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:33.005236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:33.062171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ea229e-9d00-4902-ab65-c053ecddfbd0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:33.063613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c98dd-cdab-4e81-9f2f-d5f37cc15e30 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:33.067863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:33.068518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:33.068801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:33.069618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:33.074658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:33.163705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:33.165106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:33.165568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:33.165970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:33.172268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:33.174213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:33.176716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3bad63-6172-41d6-93ef-dcbaae16d2d8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:33.190907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 701/1397] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:33.191733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:33.192887 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:33.193373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:33.193810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:33.222601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:33.223966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:33.235219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 702/1398] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:33.247061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:33.279398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e450505a-e0f4-4867-b93f-b0595749515f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:33.294769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:33.296675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3ced83-46e3-41d2-9a6e-6fb8325d7af6 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:33.297818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:33.299736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:33.302220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:33.304599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:33.324735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 697/1399] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:33 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:33.347809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:33.349695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:33.365453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 703/1400] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:34.013874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b36939-3588-4e1e-b032-4cace6d4f708 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:34.014537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b36939-3588-4e1e-b032-4cace6d4f708 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:34.014537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b36939-3588-4e1e-b032-4cace6d4f708 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:34.262075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:34.264004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:34.281115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b4b36939-3588-4e1e-b032-4cace6d4f708 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:34.290839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b4b36939-3588-4e1e-b032-4cace6d4f708 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:34.291902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d6631-f4e3-4857-8108-c6eb4e0ad009 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:34.327088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:34.327746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:34.328064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:34.328332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] PATH_INFO: `/v3/projects/fd203962e98e40dea828d8223b1d5a58` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:34.346940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd203962e98e40dea828d8223b1d5a58)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:34.351963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: WARNING py.warnings [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Oct 23 02:11:34.351963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: warnings.warn(msg) Oct 23 02:11:34.351963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: Oct 23 02:11:34.355769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:34.358412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:34.550417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd203962e98e40dea828d8223b1d5a58', 'request_id': 'req-79f0ecb4-9210-455e-bcee-ffe5895d3355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:34.567228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 698/1401] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:11:34 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 555 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:34.568589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ceb97-4c3c-41b7-939a-896c90cbf7fa None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:34.568870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ceb97-4c3c-41b7-939a-896c90cbf7fa None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:34.569153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616ceb97-4c3c-41b7-939a-896c90cbf7fa None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:34.570461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 699/1402] 10.222.0.23 () {54 vars in 912 bytes} [Sun Oct 23 02:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:34.587494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:34.587785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:34.588197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:34.602477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd203962e98e40dea828d8223b1d5a58', 'request_id': 'req-79f0ecb4-9210-455e-bcee-ffe5895d3355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:34.635870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd203962e98e40dea828d8223b1d5a58', 'request_id': 'req-79f0ecb4-9210-455e-bcee-ffe5895d3355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:34.656113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] The token cache is being invalidate because project fd203962e98e40dea828d8223b1d5a58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:34.656360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:34.657417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:34.659830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 704/1403] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:34 2022] DELETE /identity/v3/projects/fd203962e98e40dea828d8223b1d5a58 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:34.662960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:34.665040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:34.672060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:34.674553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:34.795985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0ecb4-9210-455e-bcee-ffe5895d3355 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:34.810543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:34.813857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:34.814389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:34.814820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] PATH_INFO: `/v3/users/8b93ac8ce86d41e989c3c42383beaea1` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:34.818595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b93ac8ce86d41e989c3c42383beaea1)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:34.826385 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:34.849725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 700/1404] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:11:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:34.866765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:34.868840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:34.958912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:34.993331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b93ac8ce86d41e989c3c42383beaea1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:35.016898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b93ac8ce86d41e989c3c42383beaea1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:35.034984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b93ac8ce86d41e989c3c42383beaea1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:35.035998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b93ac8ce86d41e989c3c42383beaea1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:35.036620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b93ac8ce86d41e989c3c42383beaea1'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:35.094341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 705/1405] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:34 2022] DELETE /identity/v3/users/8b93ac8ce86d41e989c3c42383beaea1 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:35.098807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:35.102600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:35.126386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb14a92-6749-4262-b0d8-ce66322fbe33 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:35.131445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:35.289014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de062b-eee6-4813-bc38-5de0e33985ee admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:35.293439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:35.295216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:35.295698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:35.296161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:35.303994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:35.313157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:35.313790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:35.314071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:35.314345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:35.318001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:35.330836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:35.333059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:35.351816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 706/1406] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:35.484623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:35.493843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:35.521326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 701/1407] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:11:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 655 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 02:11:35.552922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:35.554172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:35.819904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229bd4a2-3fb3-4cc8-b0af-152c300a3858 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:35.833810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:35.837723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:35.839414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:35.841563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] PATH_INFO: `/v3/projects/b19eadcff3d740c7bd8cc0952474a765` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:35.847230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b19eadcff3d740c7bd8cc0952474a765)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:35.855332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:35.869042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:35.870638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:35.907538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdddc98-d1f2-4581-a317-2655a4b99393 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:35.947600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:35.948338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:35.948645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:35.948949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] PATH_INFO: `/v3/projects/79275f37a32140718ec85a2551b4180f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:35.950929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=79275f37a32140718ec85a2551b4180f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:35.963618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:36.103115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b19eadcff3d740c7bd8cc0952474a765', 'request_id': 'req-56c82f46-a74c-4808-b555-df4e4ad24446'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.124994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b19eadcff3d740c7bd8cc0952474a765', 'request_id': 'req-56c82f46-a74c-4808-b555-df4e4ad24446'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.138702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b19eadcff3d740c7bd8cc0952474a765', 'request_id': 'req-56c82f46-a74c-4808-b555-df4e4ad24446'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.143787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:36.152096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '79275f37a32140718ec85a2551b4180f', 'request_id': 'req-8238ffdb-6760-4411-8465-7a735027a8ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.175685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] The token cache is being invalidate because project b19eadcff3d740c7bd8cc0952474a765 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:36.178732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.179958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.182546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '79275f37a32140718ec85a2551b4180f', 'request_id': 'req-8238ffdb-6760-4411-8465-7a735027a8ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.188218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 707/1408] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:35 2022] DELETE /identity/v3/projects/b19eadcff3d740c7bd8cc0952474a765 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:36.189442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:36.196627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:36.212165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '79275f37a32140718ec85a2551b4180f', 'request_id': 'req-8238ffdb-6760-4411-8465-7a735027a8ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.250848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] The token cache is being invalidate because project 79275f37a32140718ec85a2551b4180f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:36.251227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.252519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.256101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 702/1409] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:35 2022] DELETE /identity/v3/projects/79275f37a32140718ec85a2551b4180f => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:36.260333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:36.261762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:36.310574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:36.365910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c82f46-a74c-4808-b555-df4e4ad24446 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:36.548825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8238ffdb-6760-4411-8465-7a735027a8ab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:36.564647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:36.569276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:36.569569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:36.570854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] PATH_INFO: `/v3/users/6d3795c179574162a258093e81db6dc5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:36.576551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d3795c179574162a258093e81db6dc5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:36.579410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:36.580221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:36.580644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:36.581136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] PATH_INFO: `/v3/users/8da7fe3787f541219a1ad351c4138460` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:36.583188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8da7fe3787f541219a1ad351c4138460)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:36.584526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:36.588668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:36.723028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8da7fe3787f541219a1ad351c4138460'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.735093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8da7fe3787f541219a1ad351c4138460'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.770182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8da7fe3787f541219a1ad351c4138460'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.772412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8da7fe3787f541219a1ad351c4138460'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.775913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7928f07f-db87-4dc0-9b38-13a3995b5117 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8da7fe3787f541219a1ad351c4138460'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.804366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d3795c179574162a258093e81db6dc5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.820093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 708/1410] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:36 2022] DELETE /identity/v3/users/8da7fe3787f541219a1ad351c4138460 => generated 0 bytes in 631 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:36.839340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d3795c179574162a258093e81db6dc5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.846240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d3795c179574162a258093e81db6dc5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.876647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d3795c179574162a258093e81db6dc5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.879010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d3795c179574162a258093e81db6dc5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:36.950512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 703/1411] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:36 2022] DELETE /identity/v3/users/6d3795c179574162a258093e81db6dc5 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:37.981021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:37.982364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:37.983034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:38.034154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:38.036214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:38.212238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 709/1412] 10.222.0.49 () {58 vars in 1015 bytes} [Sun Oct 23 02:11:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:38.303173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:38.304663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:38.347305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da0b1a-32fe-4f3c-9880-a170869e72f0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:38.371515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:38.372325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:38.379220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:38.379900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:38.412487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:38.425176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a78218-33ca-4d5c-97cf-d6d0c8dd2892 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:38.450832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 704/1413] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:38.677620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:38.689512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:38.737649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:38.984634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36ac1f-264a-4e9f-b9ed-b80598a4ceec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:38.995456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:38.996223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:38.998251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:39.020967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] PATH_INFO: `/v3/projects/1d0ae24e18ee431ca4fed3a21593d648` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:39.023641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d0ae24e18ee431ca4fed3a21593d648)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:39.040861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:39.244863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d0ae24e18ee431ca4fed3a21593d648', 'request_id': 'req-53dc825d-67d9-4e32-9b5f-3c057496aa70'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.269767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d0ae24e18ee431ca4fed3a21593d648', 'request_id': 'req-53dc825d-67d9-4e32-9b5f-3c057496aa70'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.310244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d0ae24e18ee431ca4fed3a21593d648', 'request_id': 'req-53dc825d-67d9-4e32-9b5f-3c057496aa70'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.336408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] The token cache is being invalidate because project 1d0ae24e18ee431ca4fed3a21593d648 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:39.337053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.338286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-53dc825d-67d9-4e32-9b5f-3c057496aa70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.343403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 710/1414] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:38 2022] DELETE /identity/v3/projects/1d0ae24e18ee431ca4fed3a21593d648 => generated 0 bytes in 666 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:39.348354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:39.348949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:39.348949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:39.408718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:39.413282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:39.518126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:39.636262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 705/1415] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:39.640925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:39.641224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:39.641485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:39.670077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:39.671670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:39.705235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:39.706497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:39.726625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2d49b8-45f0-4183-a03d-87ae68bee2af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:39.749517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:39.750275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:39.754186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:39.754546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] PATH_INFO: `/v3/projects/6500f18ae566427bbdd5e65fcc390fda` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:39.758182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6500f18ae566427bbdd5e65fcc390fda)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:39.759964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 711/1416] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:39.764407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:39.766094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:39.802867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017b51a-fa9a-4a24-8003-8782f1f70561 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:39.851372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:39.852007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:39.852333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:39.852638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:39.853377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:39.854131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:39.866053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:39.949153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6500f18ae566427bbdd5e65fcc390fda', 'request_id': 'req-71b95444-6a6f-4cbb-9974-ab6d62bb0218'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:39.975061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 712/1417] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:39.977155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:39.983605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6500f18ae566427bbdd5e65fcc390fda', 'request_id': 'req-71b95444-6a6f-4cbb-9974-ab6d62bb0218'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.009431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6500f18ae566427bbdd5e65fcc390fda', 'request_id': 'req-71b95444-6a6f-4cbb-9974-ab6d62bb0218'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.030973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] The token cache is being invalidate because project 6500f18ae566427bbdd5e65fcc390fda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:40.031261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.031590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5405b3bf-8dd2-4028-b61b-d85df8e2c99c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:40.032628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-71b95444-6a6f-4cbb-9974-ab6d62bb0218 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.034686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 706/1418] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:39 2022] DELETE /identity/v3/projects/6500f18ae566427bbdd5e65fcc390fda => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:40.039677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:40.040188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:40.040622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:40.085704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:40.095132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:40.370657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:40.373702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:40.374213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:40.374345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:40.379049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:40.386624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 707/1419] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:40.389953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:40.392713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:40.400483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785a46e5-c597-4c61-8709-127736e6d053 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:40.459192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c5d79f-d286-4e34-96b2-a4b4707422ed None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:40.485330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:40.486475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:40.486758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:40.487049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] PATH_INFO: `/v3/projects/064bf192c42b4d82a89e9591788b72ae` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:40.488230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=064bf192c42b4d82a89e9591788b72ae)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:40.494851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:40.507040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 713/1420] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:39 2022] POST /identity/v3/projects => generated 385 bytes in 528 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:40.507040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:40.507040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:40.511096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:40.619768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] MFA Rules not processed for user `2335878ea1d045858b158f6b4175e896`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:40.623883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:40.662840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:40.670313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '064bf192c42b4d82a89e9591788b72ae', 'request_id': 'req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.690365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '064bf192c42b4d82a89e9591788b72ae', 'request_id': 'req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.704340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:40.710970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '064bf192c42b4d82a89e9591788b72ae', 'request_id': 'req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.744624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] The token cache is being invalidate because project 064bf192c42b4d82a89e9591788b72ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:40.745317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.749209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aa4bbe43-b733-4c08-a115-43ec35ff5ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:40.752795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 708/1421] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:40 2022] DELETE /identity/v3/projects/064bf192c42b4d82a89e9591788b72ae => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:40.754553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e2fe3-7d17-4d56-8fd7-be8fcea181a0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:40.756629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e2fe3-7d17-4d56-8fd7-be8fcea181a0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:40.756629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2e2fe3-7d17-4d56-8fd7-be8fcea181a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:40.820478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 714/1422] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:40 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:40.822711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:40.825670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7e2e2fe3-7d17-4d56-8fd7-be8fcea181a0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:40.828311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:40.833526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7e2e2fe3-7d17-4d56-8fd7-be8fcea181a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:41.220441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c4c96a-2935-4e2e-a26c-f5b7da9b71c2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.225814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 709/1423] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 470 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:41.234526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.235643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.236174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.236588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.237952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.240156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.242155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.242747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.243100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.392321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 715/1424] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:40 2022] POST /identity/v3/users => generated 293 bytes in 571 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:41.405161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:41.406569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:41.407857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.466100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e212406-2340-47a9-9877-6599c07d4d21 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.466268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 710/1425] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:41.466268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.508922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.509556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.512029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.512579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.512975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2187da28-0464-4e35-9c0a-1789904c025c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.513080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:41.525870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.531644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2662b1a-de10-49e7-b086-3d5e4ccb6c45 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.536638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.537266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.537551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.537829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.538602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.542695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.611958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 716/1426] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:41.613460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.619871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.623111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.628139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 711/1427] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:41.629097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.670208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a87fb-9364-45b2-8461-72f5b12e78b3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.710429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:41.713900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:41.714079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.714666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.715493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.715785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.733463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.738359 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 717/1428] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:41.739539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.743693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.770155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21080f4-47ef-41d0-bce5-6f6ae249463f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.784797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.785646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.786688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.787122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.787959 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.789903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.827767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 718/1429] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:41.829438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.887826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b798e-ade9-4952-a911-2d4808003b88 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:41.904902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 712/1430] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:41 2022] POST /identity/v3/projects => generated 407 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:41.915333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:41.915966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:41.916219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:41.916490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:41.918826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:41.919335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:41.920212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:41.924764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:41.981119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 719/1431] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:41.989062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:42.016110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec829a4f-b378-4137-834c-e8fc8fdca31c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:42.224267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:42.226407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:42.226924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:42.227431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:42.230840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:42.235786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:42.277914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:42.326188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b231b8-58e0-442c-aa3d-89dfb99eaf2b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:42.326811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 720/1432] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:41 2022] POST /identity/v3/users => generated 304 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:42.326811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:42.334467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:42.335270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:42.335605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:42.335935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] PATH_INFO: `/v3/projects/08ac0ebbd9af4d0989b90f290712dde4/users/382cfabc10e34d3a90b7422a4dcb7e42/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:42.369462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c669ac-c3f8-4aaa-8615-a7a2bef84d47 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:42.383520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ac0ebbd9af4d0989b90f290712dde4, user_id=382cfabc10e34d3a90b7422a4dcb7e42, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:42.392122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:42.393056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:42.394505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:42.406688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:42.408930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:42.409848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:42.427116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:42.442249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 713/1433] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:41 2022] PUT /identity/v3/projects/08ac0ebbd9af4d0989b90f290712dde4/users/382cfabc10e34d3a90b7422a4dcb7e42/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 522 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:42.448264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:42.536067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 721/1434] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:42 2022] POST /identity/v3/projects => generated 365 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:42.536067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:42.542194 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:42.592092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:42.674056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0be26f-57c7-43fe-a4c9-be9b39d855ff admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:42.816269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9f9f9d-35db-4077-b62b-f1fe860155c2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:42.836710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:42.837379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:42.837658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:42.837932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] PATH_INFO: `/v3/users/b99ef823f08140d69edb54a7826b4763` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:42.839262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99ef823f08140d69edb54a7826b4763)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:42.851651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:42.937858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:42.937858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:42.938219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:42.938567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:42.940104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:42.962535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cc6010-025f-4c6f-a413-c3df309135fe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:43.009574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 714/1435] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 567 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:43.011855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.012177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.012644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.020061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99ef823f08140d69edb54a7826b4763'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:43.052904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99ef823f08140d69edb54a7826b4763'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:43.093874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99ef823f08140d69edb54a7826b4763'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:43.094617 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99ef823f08140d69edb54a7826b4763'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:43.095275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99ef823f08140d69edb54a7826b4763'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:43.138014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] MFA Rules not processed for user `382cfabc10e34d3a90b7422a4dcb7e42`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:43.141482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:43.158233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 722/1436] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:42 2022] DELETE /identity/v3/users/b99ef823f08140d69edb54a7826b4763 => generated 0 bytes in 624 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:43.183043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:43.187703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:43.372669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:43.409431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81994f24-5f4b-4894-b3a5-01a46ecfc5f0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:43.430626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:43.431546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.432910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.433304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.437098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:43.440506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b27ccc-b41b-42af-b5ce-ea9035b24b15 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:43.455386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 715/1437] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 445 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:43.456914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:43.467570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:43.503754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b243cc2-d194-4d9f-bdd5-4a7c26c171e7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:43.511302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:43.513557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.513996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.514638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.515781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:43.517886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5341a7-c364-467e-8097-3466be14efe1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:43.532125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 716/1438] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:43.534413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.535229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.535652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.545641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 723/1439] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:43 2022] POST /identity/v3/users => generated 283 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:43.560526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.565781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.566931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.576962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 724/1440] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:11:43 2022] GET /identity => generated 270 bytes in 31 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:43.577992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:43.578167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:43.581756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:43.587265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:43.606074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 717/1441] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:43.606074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:43.632841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6330543-d73a-45c2-a14f-23a05cfc4647 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:43.644704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:43.645445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.645735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.646474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] PATH_INFO: `/v3/projects/dc69877a9a0c494886d525c40e6bcf06/users/6246d4721a3546e88d118daf8e3fa78e/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.698056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a83079d-bc5c-40b1-80ff-35c154aecc16 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:43.703635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc69877a9a0c494886d525c40e6bcf06, user_id=6246d4721a3546e88d118daf8e3fa78e, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:43.729760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:43.732101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:43.732723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.733009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.733275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.733906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:43.735829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8cdf60-52dc-4c35-8f4e-59422f3353b7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:43.758806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 718/1442] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:43 2022] PUT /identity/v3/projects/dc69877a9a0c494886d525c40e6bcf06/users/6246d4721a3546e88d118daf8e3fa78e/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:43.759855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:43.842135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:43.877697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 725/1443] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:43 2022] POST /identity/v3/projects => generated 385 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:43.881693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a5ab-46bd-4f6d-85b6-8ec91f3e9b7e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:43.884167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a5ab-46bd-4f6d-85b6-8ec91f3e9b7e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:43.884474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90a5ab-46bd-4f6d-85b6-8ec91f3e9b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:43.959525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2c90a5ab-46bd-4f6d-85b6-8ec91f3e9b7e None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:43.960995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c90a5ab-46bd-4f6d-85b6-8ec91f3e9b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:44.004576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9439120d-5c51-4bda-96bc-3ad25e578ccb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.139511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.140517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.140799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.141191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.143096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.145135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.156647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 726/1444] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:11:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:44.159527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.159839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.160100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 719/1445] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:44.160213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.161190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.184672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c1967-5fa3-4ce7-8dcd-24bf8ded40bb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.195494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.196497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.196933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.197522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.200684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.203688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.242006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] MFA Rules not processed for user `6246d4721a3546e88d118daf8e3fa78e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:44.244218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:44.253912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 720/1446] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:44 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:44.257844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.261241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:44.435288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695a8e76-7d5e-4950-ad1f-010a10c9fa0d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.443717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.445100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.445561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.446213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.465473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.468842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.477673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 727/1447] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:44 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:44.478691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.498710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 721/1448] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:44.503087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.526848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae33f07-adda-4de3-9a23-fd9657b246c8 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.527129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50b7834-168a-4e00-a0fb-cd8651aef8e6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.540665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.541472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.541920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.542277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.543303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.544137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.545616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.546128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.546636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.548938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.552126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.557419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.577470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 722/1449] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:44.578833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.582019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 728/1450] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:44.582916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.583482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:44.626491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2caeead-2af4-4cb9-b691-fa73f2de0728 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.631507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacc51b7-cb6d-4b90-87b5-441a7ae2c61e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.644819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.645820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.646302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.646951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.647332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.647585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.647913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.648701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.653817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.656038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.661007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.662868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.702260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 723/1451] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:44.705308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.735176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c9f99-27c2-45d2-91cd-52db43cfcebc service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.739794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 729/1452] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:44 2022] POST /identity/v3/projects => generated 407 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:44.742573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:44.770218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:44.815560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:44.882760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.897502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3df073-6f4a-4b74-b9bb-26395514e6e3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:44.898552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.898992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.899430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] PATH_INFO: `/v3/projects/ef926d786c404795ae9eeb5ac5404a8b/users/48c79576784c4e77be16d25950223e7b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.912271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:44.913231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:44.914066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:44.916044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] PATH_INFO: `/v3/projects/e84e0df0e0444223a53bfe70ea0d7cc0/users/0f86a65daf43470e82656c6d8fddfd43/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:44.949656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef926d786c404795ae9eeb5ac5404a8b, user_id=48c79576784c4e77be16d25950223e7b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.957847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.966480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84e0df0e0444223a53bfe70ea0d7cc0, user_id=0f86a65daf43470e82656c6d8fddfd43, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:44.973856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:44.996322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 724/1453] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:44 2022] PUT /identity/v3/projects/ef926d786c404795ae9eeb5ac5404a8b/users/48c79576784c4e77be16d25950223e7b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:44.996322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.014375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 730/1454] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:44 2022] PUT /identity/v3/projects/e84e0df0e0444223a53bfe70ea0d7cc0/users/0f86a65daf43470e82656c6d8fddfd43/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:45.016529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.200436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95fe43d-3302-4c0c-9a26-54efe1dc6d74 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.212045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cc40a-3ced-475f-9ef1-9a780f480111 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.226678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:45.228217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.229476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.229763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.231293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:45.232407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:45.232850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.233209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.234539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.235422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:45.236973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cffdfa-b0d7-4bd3-b8d5-bb971cc6572e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:45.242197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:45.292211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 725/1455] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:44 2022] POST /identity/v3/users => generated 310 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:45.292211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.292211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.293731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 731/1456] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:45.294561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.294796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.348927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f46026-9110-4dcb-90ec-867dd871bbaf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.368748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] MFA Rules not processed for user `0f86a65daf43470e82656c6d8fddfd43`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:45.370660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:45.374145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:45.376403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.378868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.379518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.396930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:45.403463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:45.428660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 732/1457] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:45.428660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.525118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 726/1458] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:45.526002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.527903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4379279-fa5d-4c02-8ae2-35bad07c22c5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.560366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:45.563293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.563817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.564768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] PATH_INFO: `/v3/projects/ef926d786c404795ae9eeb5ac5404a8b/users/48c79576784c4e77be16d25950223e7b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.579130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef926d786c404795ae9eeb5ac5404a8b, user_id=48c79576784c4e77be16d25950223e7b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:45.587781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f0ba64-c1a5-49f5-8482-9f7ab82b0c32 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.594197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:45.598974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:45.599842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.603786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.604175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] PATH_INFO: `/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2/users/619c3e32fe4f40ff949eba15c994bf34/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.661540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebd0f2a123a42f4bc6809f837510eb2, user_id=619c3e32fe4f40ff949eba15c994bf34, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:45.674480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f88b4b-8bd9-43ed-ace1-2c928baa5f5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:45.699855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 733/1459] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:45 2022] PUT /identity/v3/projects/ef926d786c404795ae9eeb5ac5404a8b/users/48c79576784c4e77be16d25950223e7b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:45.701486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:45.702958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 727/1460] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:45 2022] PUT /identity/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2/users/619c3e32fe4f40ff949eba15c994bf34/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:45.704408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:45.704788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:45.710680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:45.729202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:45.781941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] MFA Rules not processed for user `48c79576784c4e77be16d25950223e7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:45.783440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:45.836700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ddbc1-3119-4cf2-a327-d2853d7f7d7c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:45.844443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:45.967584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 728/1461] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:45 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:45.968548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:46.043415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:46.043415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.043415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.043785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.044349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:46.046298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:46.109661 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 734/1462] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:45 2022] POST /identity/v3/projects => generated 369 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:46.119705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:46.133755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:46.246199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a225d899-b9c4-46cd-840e-2b671649a4b6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:46.291182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ff3428-c58a-400d-a3bc-1a30239b5885 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:46.334894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:46.335540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.335827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.336109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.476200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:46.477203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.480883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.483033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.483986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:46.490913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:46.533622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 729/1463] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:45 2022] GET /identity/v3/roles => generated 1420 bytes in 565 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:46.542139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:46.618326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfb7b9-c92d-4152-91f6-4013d84998e1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:46.663503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:46.670108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:46.683765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:46.684727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.691158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.691914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.693237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:46.697990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:46.728299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 735/1464] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:46 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 618 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:46.729122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:46.744981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad6f7b-ca0d-4a2d-8d89-e749ed4ce828 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:46.752721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:46.753421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.754080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.754777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] PATH_INFO: `/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2/users/619c3e32fe4f40ff949eba15c994bf34/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.758704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ebd0f2a123a42f4bc6809f837510eb2, user_id=619c3e32fe4f40ff949eba15c994bf34, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:46.762213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9126d7e-7bac-41a7-abe6-3d167ee3ce08 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:46.798567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 736/1465] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:46 2022] PUT /identity/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2/users/619c3e32fe4f40ff949eba15c994bf34/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:46.820762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:46.821557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:46.822333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:46.869073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 730/1466] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:46 2022] POST /identity/v3/users => generated 291 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:46.872109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:46.895591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] MFA Rules not processed for user `619c3e32fe4f40ff949eba15c994bf34`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:46.896878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:46.900286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b044c0d-cbde-4e8a-b498-971c8eb52d12 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:46.929584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:46.966003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:47.113113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 737/1467] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:46 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:47.114395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:47.298367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d64f9-36ff-4ce4-ba07-abad04494bd9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:47.307341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:47.307966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.308385 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.308897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.310068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:47.312547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:47.343919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 738/1468] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:47.343919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:47.348737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:47.349450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.349740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.350021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] PATH_INFO: `/v3/users/382cfabc10e34d3a90b7422a4dcb7e42` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.351261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] RBAC: Authorizing `identity:delete_user(user_id=382cfabc10e34d3a90b7422a4dcb7e42)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:47.353369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:47.463365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '382cfabc10e34d3a90b7422a4dcb7e42'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:47.477872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366681-4053-4c5e-9b73-8f36b0d0d38a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:47.485277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:47.486086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.486377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.487013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.492010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '382cfabc10e34d3a90b7422a4dcb7e42'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:47.492980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '382cfabc10e34d3a90b7422a4dcb7e42'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:47.512246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '382cfabc10e34d3a90b7422a4dcb7e42'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:47.513039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '382cfabc10e34d3a90b7422a4dcb7e42'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:47.540848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:47.542755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f944d986-bbf9-473e-a44a-df6202549f95 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:47.559425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 731/1469] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:46 2022] DELETE /identity/v3/users/382cfabc10e34d3a90b7422a4dcb7e42 => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:47.559425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:47.559425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:47.567490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 739/1470] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:47 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:47.571191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.572869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.573331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.626786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:47.628170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:47.725130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c4f14b-448e-4af4-a581-776a6b784a9e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:47.741542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:47.741893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.742197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.742518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] PATH_INFO: `/v3/projects/e2e1e4bc2f66496bb78779604ef11237/users/cabbc2615d5f40c08a72a1dd61416ee5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.771177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e1e4bc2f66496bb78779604ef11237, user_id=cabbc2615d5f40c08a72a1dd61416ee5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:47.774317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:47.805082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 732/1471] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:47 2022] PUT /identity/v3/projects/e2e1e4bc2f66496bb78779604ef11237/users/cabbc2615d5f40c08a72a1dd61416ee5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:47.809843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:47.812527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:47.847603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 740/1472] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:47.863026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:47.936773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:47.940155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:47.950056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d705870-aa2a-458e-9552-915d5e15f10f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:47.964731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:47.965703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:47.967398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:47.967713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:47.968862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:47.972074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:48.006868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 733/1473] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:48.011084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:48.031427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7c608-3b81-438a-9074-4ee7baea790c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:48.044339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:48.045165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:48.045573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:48.048317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] PATH_INFO: `/v3/projects/e2e1e4bc2f66496bb78779604ef11237/users/cabbc2615d5f40c08a72a1dd61416ee5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:48.055263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2e1e4bc2f66496bb78779604ef11237, user_id=cabbc2615d5f40c08a72a1dd61416ee5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:48.060110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2314d0b-0f6d-486e-ac98-29370067e6f9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:48.091286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 734/1474] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:48 2022] PUT /identity/v3/projects/e2e1e4bc2f66496bb78779604ef11237/users/cabbc2615d5f40c08a72a1dd61416ee5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:48.113100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:48.113624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:48.117894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:48.118128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926705bc-fcb6-4eed-b3f8-522013dce019 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:48.156501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] MFA Rules not processed for user `cabbc2615d5f40c08a72a1dd61416ee5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:48.158246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:48.275186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:48.280345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:48.280830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:48.281350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:48.326305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 735/1475] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:48.327687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:48.344185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:48.457766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:48.460195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:48.462560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e151f8-e2a4-4db6-981e-4a6f33c048b1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:48.479750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:48.483110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:48.484235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:48.484908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:48.487709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 741/1476] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 551 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:48.488904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:48.491854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:48.498102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:48.527646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f950a04b-e9a2-4af4-ba38-377c71d6e645 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:48.537343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:48.537977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:48.538260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:48.540732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:48.562321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:48.563999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:48.592371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 742/1477] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:48.640600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:48.693168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:48.706047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:48.723834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 736/1478] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:48 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:49.847597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:49.850434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:49.889104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46a95fc-88a9-44fa-88cb-880a1fdc7ed0 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:49.923439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:49.925516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:49.926315 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:49.926827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] PATH_INFO: `/v3/projects/08ac0ebbd9af4d0989b90f290712dde4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:49.930033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ac0ebbd9af4d0989b90f290712dde4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:49.935141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:50.083742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:50.095716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ac0ebbd9af4d0989b90f290712dde4', 'request_id': 'req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.104347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ac0ebbd9af4d0989b90f290712dde4', 'request_id': 'req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.170406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ac0ebbd9af4d0989b90f290712dde4', 'request_id': 'req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.213263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] The token cache is being invalidate because project 08ac0ebbd9af4d0989b90f290712dde4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:50.213847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.238329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.244946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 743/1479] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:49 2022] DELETE /identity/v3/projects/08ac0ebbd9af4d0989b90f290712dde4 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:50.251036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:50.299061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b66d3d-f34d-471a-bdc4-a16e3e6e77c2 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:50.522065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:50.522750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:50.523029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:50.523297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] PATH_INFO: `/v3/users/48c79576784c4e77be16d25950223e7b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:50.524889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48c79576784c4e77be16d25950223e7b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:50.528115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:50.692952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48c79576784c4e77be16d25950223e7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.721373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48c79576784c4e77be16d25950223e7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.723724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48c79576784c4e77be16d25950223e7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.741005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48c79576784c4e77be16d25950223e7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.741878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48c79576784c4e77be16d25950223e7b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:50.778657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 737/1480] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:50 2022] DELETE /identity/v3/users/48c79576784c4e77be16d25950223e7b => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:51.052243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:51.054726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:51.084632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814607c1-d0c5-4ca8-ac1c-2e77440b3c90 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:51.307026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:51.308249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:51.312818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:51.313125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] PATH_INFO: `/v3/projects/8eebb5bda00044c2b1c5bfa0078ab047` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:51.315401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eebb5bda00044c2b1c5bfa0078ab047)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:51.319614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:51.448670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eebb5bda00044c2b1c5bfa0078ab047', 'request_id': 'req-c511eaf1-e05c-4e6d-9e69-991609ac68e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.476213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eebb5bda00044c2b1c5bfa0078ab047', 'request_id': 'req-c511eaf1-e05c-4e6d-9e69-991609ac68e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.524589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eebb5bda00044c2b1c5bfa0078ab047', 'request_id': 'req-c511eaf1-e05c-4e6d-9e69-991609ac68e8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.546833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] The token cache is being invalidate because project 8eebb5bda00044c2b1c5bfa0078ab047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:51.547425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.549069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.552001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 744/1481] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:51 2022] DELETE /identity/v3/projects/8eebb5bda00044c2b1c5bfa0078ab047 => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:51.555890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:51.559935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:51.587385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8616d0ad-6521-4f09-9fc2-b0a7ccea3c39 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:51.676904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:51.680213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:51.680213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:51.680213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] PATH_INFO: `/v3/users/50a2bbb6bef043ba906d9e0775e9c285` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:51.680213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] RBAC: Authorizing `identity:delete_user(user_id=50a2bbb6bef043ba906d9e0775e9c285)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:51.686316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:51.701636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:51.765954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50a2bbb6bef043ba906d9e0775e9c285'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.775711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50a2bbb6bef043ba906d9e0775e9c285'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.777752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50a2bbb6bef043ba906d9e0775e9c285'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.791387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50a2bbb6bef043ba906d9e0775e9c285'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.791387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b73c52fc-9bc1-410d-b702-700058b9e10d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50a2bbb6bef043ba906d9e0775e9c285'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:51.831954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 738/1482] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:51 2022] DELETE /identity/v3/users/50a2bbb6bef043ba906d9e0775e9c285 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:51.838319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:51.840856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:51.860861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511eaf1-e05c-4e6d-9e69-991609ac68e8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:51.968219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:51.969133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:51.969526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:51.970115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] PATH_INFO: `/v3/users/0f86a65daf43470e82656c6d8fddfd43` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:51.971377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f86a65daf43470e82656c6d8fddfd43)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:51.973435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:51.977744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:52.050861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f86a65daf43470e82656c6d8fddfd43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.059634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f86a65daf43470e82656c6d8fddfd43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.070065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f86a65daf43470e82656c6d8fddfd43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.071584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f86a65daf43470e82656c6d8fddfd43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.072498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f86a65daf43470e82656c6d8fddfd43'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.094730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 745/1483] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:51 2022] DELETE /identity/v3/users/0f86a65daf43470e82656c6d8fddfd43 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:52.099030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.099410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.099778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.119656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:52.121674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.212057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.213651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.270049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 739/1484] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:52.284944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.286507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.389239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e1834-95ac-46fc-9ff5-04da0a27cbfb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.393373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.393500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0923846a-f41d-4f23-b1b4-e65967ee3f42 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.394490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.394810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.395183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.398475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.399124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.399549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.399836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] PATH_INFO: `/v3/projects/ef926d786c404795ae9eeb5ac5404a8b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.401571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef926d786c404795ae9eeb5ac5404a8b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.405398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.405779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.407357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.420592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 740/1485] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:52.421400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.423156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.437783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfe4ac2-6901-402b-9937-bb57e2947146 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.443960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.444981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.445671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.446138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] PATH_INFO: `/v3/projects/4804846d26ce49df866f808594564022` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.448909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] RBAC: Authorizing `identity:delete_project(project_id=4804846d26ce49df866f808594564022)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.451511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.474608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:52.496909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef926d786c404795ae9eeb5ac5404a8b', 'request_id': 'req-14fde548-713c-4948-b07e-d52658329866'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.509107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef926d786c404795ae9eeb5ac5404a8b', 'request_id': 'req-14fde548-713c-4948-b07e-d52658329866'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.520096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef926d786c404795ae9eeb5ac5404a8b', 'request_id': 'req-14fde548-713c-4948-b07e-d52658329866'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.541445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4804846d26ce49df866f808594564022', 'request_id': 'req-dfb681b5-eef6-4412-bef2-9f4dc273d6af'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.554636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4804846d26ce49df866f808594564022', 'request_id': 'req-dfb681b5-eef6-4412-bef2-9f4dc273d6af'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.564735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] The token cache is being invalidate because project ef926d786c404795ae9eeb5ac5404a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:52.565052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.565539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4804846d26ce49df866f808594564022', 'request_id': 'req-dfb681b5-eef6-4412-bef2-9f4dc273d6af'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.568183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-14fde548-713c-4948-b07e-d52658329866 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.578373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 746/1486] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:52 2022] DELETE /identity/v3/projects/ef926d786c404795ae9eeb5ac5404a8b => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:52.580422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8f68-13d9-4227-9f73-66e53ff9bc5f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.581924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8f68-13d9-4227-9f73-66e53ff9bc5f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.582266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44e8f68-13d9-4227-9f73-66e53ff9bc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.596692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] The token cache is being invalidate because project 4804846d26ce49df866f808594564022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:52.597136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.598791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dfb681b5-eef6-4412-bef2-9f4dc273d6af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:52.604644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 741/1487] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:52 2022] DELETE /identity/v3/projects/4804846d26ce49df866f808594564022 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:52.611505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7540-3ce6-43a3-beab-f05926f1c1ab None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.612011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7540-3ce6-43a3-beab-f05926f1c1ab None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.612485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6e7540-3ce6-43a3-beab-f05926f1c1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.635650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c44e8f68-13d9-4227-9f73-66e53ff9bc5f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:52.637593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c44e8f68-13d9-4227-9f73-66e53ff9bc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.641121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2a6e7540-3ce6-43a3-beab-f05926f1c1ab None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:52.642834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a6e7540-3ce6-43a3-beab-f05926f1c1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.654989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:52.768059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 742/1488] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:52.779291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.779590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.780022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.810748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 747/1489] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:52.818725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab879-a34d-4beb-a323-519585210960 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.821511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:52.822017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab879-a34d-4beb-a323-519585210960 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.822587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265ab879-a34d-4beb-a323-519585210960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.827136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.850890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 743/1490] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:52.858077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.872166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-265ab879-a34d-4beb-a323-519585210960 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:52.873907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-265ab879-a34d-4beb-a323-519585210960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.880928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0217cd9f-cf28-4aeb-af03-2a93c6cb9c18 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.896459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.897172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.897450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.897760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.898482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:52.898810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.901847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.907886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 748/1491] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:52.910822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab879-a34d-4beb-a323-519585210960 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.923335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265ab879-a34d-4beb-a323-519585210960 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.925455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 744/1492] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:52.926742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.929250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:52.930520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.931343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.931937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.932368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.933976 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:52.934483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.936370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.952110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b6a38-c6d5-4dfc-ba06-420e1f71712a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.953980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 749/1493] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:52.955235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:52.961896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.962942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.964728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.965117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] PATH_INFO: `/v3/projects/e84e0df0e0444223a53bfe70ea0d7cc0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.967927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84e0df0e0444223a53bfe70ea0d7cc0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.968836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de32524-2feb-432f-8ca3-bb40ce22e925 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:52.970061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:52.975546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:52.975726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:52.976062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:52.976342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:52.977192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:52.978940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.011755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 750/1494] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:52 2022] POST /identity/v3/projects => generated 367 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:53.012733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.052048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84e0df0e0444223a53bfe70ea0d7cc0', 'request_id': 'req-d7a414c2-071d-4e3f-aabb-480fdd6875a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.069481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84e0df0e0444223a53bfe70ea0d7cc0', 'request_id': 'req-d7a414c2-071d-4e3f-aabb-480fdd6875a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.079240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84e0df0e0444223a53bfe70ea0d7cc0', 'request_id': 'req-d7a414c2-071d-4e3f-aabb-480fdd6875a3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.097405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] The token cache is being invalidate because project e84e0df0e0444223a53bfe70ea0d7cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:53.097795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.098472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.100420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 745/1495] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:52 2022] DELETE /identity/v3/projects/e84e0df0e0444223a53bfe70ea0d7cc0 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:53.101244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.102660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.107996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414d44e2-c10e-4be4-a975-b562bda9ea79 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.114829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:53.185233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.185988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.186405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.186794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.187913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.189449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.191996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a414c2-071d-4e3f-aabb-480fdd6875a3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.198924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.199850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.199850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.200111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.201123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.203664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.230015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 746/1496] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/users => generated 284 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:53.231540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.232377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 751/1497] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:53.233925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.235792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.239596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:53.254628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56fa7f1-33a5-4d7a-94d0-549169f0bc77 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.327209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.328043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.328328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.329148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] PATH_INFO: `/v3/users/cabbc2615d5f40c08a72a1dd61416ee5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.330354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cabbc2615d5f40c08a72a1dd61416ee5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.332335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.332465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f667f67-c302-4b7b-bf39-b4aab0ca700f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.337014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.337605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.337888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.338164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.338872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.340464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.351411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 752/1498] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:53.352190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.353908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.370974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8172ba3b-4213-477c-8323-ff028fe95274 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.376526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.377133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.377438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.377994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.378340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.380768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.387194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cabbc2615d5f40c08a72a1dd61416ee5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.398219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cabbc2615d5f40c08a72a1dd61416ee5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.399998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cabbc2615d5f40c08a72a1dd61416ee5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.409227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 753/1499] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/users => generated 284 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:53.412941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cabbc2615d5f40c08a72a1dd61416ee5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.413609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.414115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cabbc2615d5f40c08a72a1dd61416ee5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:53.415851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.435248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2974f39-2ec6-4571-a766-f6813dccd3c5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.442178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 747/1500] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:53 2022] DELETE /identity/v3/users/cabbc2615d5f40c08a72a1dd61416ee5 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:53.442668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.442921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.443290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.443802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.444086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.444662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.444803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.446382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.460188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 754/1501] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:53.464858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.477902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e0e1bc-eb87-44f4-9dc2-f20627770888 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.493588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:53.561728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6983fd20-1835-4743-b87a-8a1bb767b962 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.568678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.570925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.571435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.571970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.574117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.576525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.584110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.584854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.585339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.585615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] PATH_INFO: `/v3/projects/e7d8654164f34ca49f73ba503a0caa8f/users/27c7d5a35c8746c09b4edf72b19e5721/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.599011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 748/1502] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:53.600779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.607363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7d8654164f34ca49f73ba503a0caa8f, user_id=27c7d5a35c8746c09b4edf72b19e5721, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.611698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.619118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc0bb52-ccf0-4953-ae1e-2b21c47ec530 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.627541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.629568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.630853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.631794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.632478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.634118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b78dab-de28-4b79-964e-29600529caf4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.642122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 755/1503] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:53 2022] PUT /identity/v3/projects/e7d8654164f34ca49f73ba503a0caa8f/users/27c7d5a35c8746c09b4edf72b19e5721/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:53.645667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 749/1504] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:53.649942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.653138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.653486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.653768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.688605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] MFA Rules not processed for user `27c7d5a35c8746c09b4edf72b19e5721`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:53.690158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.723539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:53.737462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d83977-3585-471e-ae81-cd0de16fcfc8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.743564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.744420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.744751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.745559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] PATH_INFO: `/v3/projects/1bcca70f71ad4565b50ae565c4f69195/users/e609856fc9074514a6aeac24a3a6fe19/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.789961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bcca70f71ad4565b50ae565c4f69195, user_id=e609856fc9074514a6aeac24a3a6fe19, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.795100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb401e96-63c8-40ef-83c1-578226c40f58 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.798742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 750/1505] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:53.803543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.830510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 756/1506] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:53 2022] PUT /identity/v3/projects/1bcca70f71ad4565b50ae565c4f69195/users/e609856fc9074514a6aeac24a3a6fe19/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:53.831383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae08a54a-3b6f-40ca-a922-a5c22c978175 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.832228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.832429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.832711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.878416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] MFA Rules not processed for user `e609856fc9074514a6aeac24a3a6fe19`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:53.879801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.925700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:53.926324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:53.926652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:53.926928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:53.927560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:53.929721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:53.957665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 751/1507] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/projects => generated 369 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:53.958558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.960269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:53.961651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 757/1508] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:53.962939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:53.980366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6734aacf-4288-4aa4-84ab-895c6e324dae None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:53.987811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:54.043051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a190d3-4774-4fd1-84ce-69d5d8fe981f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.047859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.048598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.049085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.049494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] PATH_INFO: `/v3/projects/e2e1e4bc2f66496bb78779604ef11237` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.050958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2e1e4bc2f66496bb78779604ef11237)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.052857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.068826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.069978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.070680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.070967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.071864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.074545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.103990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 758/1509] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:53 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:54.104919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.106630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.124260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2e1e4bc2f66496bb78779604ef11237', 'request_id': 'req-b2840e7d-762b-4018-80a5-a44ad6a1a50a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:54.134208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2e1e4bc2f66496bb78779604ef11237', 'request_id': 'req-b2840e7d-762b-4018-80a5-a44ad6a1a50a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:54.141807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2e1e4bc2f66496bb78779604ef11237', 'request_id': 'req-b2840e7d-762b-4018-80a5-a44ad6a1a50a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:54.159304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] The token cache is being invalidate because project e2e1e4bc2f66496bb78779604ef11237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:54.159618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:54.161032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b2840e7d-762b-4018-80a5-a44ad6a1a50a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:54.163764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 752/1510] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:53 2022] DELETE /identity/v3/projects/e2e1e4bc2f66496bb78779604ef11237 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:54.164949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423313-12c7-4665-a046-881f663dadc4 None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.165311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423313-12c7-4665-a046-881f663dadc4 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.165658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82423313-12c7-4665-a046-881f663dadc4 None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.167930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 753/1511] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:11:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:11:54.169331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82423313-12c7-4665-a046-881f663dadc4 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.171601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82423313-12c7-4665-a046-881f663dadc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.180508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:54.227696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091e26a6-91ea-4079-b4d1-e0ec827730ba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.266039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82423313-12c7-4665-a046-881f663dadc4 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.271464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.272084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.272368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.272645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.273544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.275342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a908302-9fdf-4f0d-86fa-72ce82f6399e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.288407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 754/1512] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:54.289759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540011-8fec-430f-8eb4-79037157aed9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.290051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540011-8fec-430f-8eb4-79037157aed9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.290324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e540011-8fec-430f-8eb4-79037157aed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.298650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.299927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.300408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.300689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.302511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.313472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4e540011-8fec-430f-8eb4-79037157aed9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:54.314724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4e540011-8fec-430f-8eb4-79037157aed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.331850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 755/1513] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:54.333794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46977-1053-4ca6-9922-4405405280cc None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.334281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46977-1053-4ca6-9922-4405405280cc None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.335260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46977-1053-4ca6-9922-4405405280cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.359058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-58a46977-1053-4ca6-9922-4405405280cc None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:54.360351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-58a46977-1053-4ca6-9922-4405405280cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.390902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.392470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.401066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:54.403580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 759/1514] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:54.404962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.421755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5fe92-4cf0-4492-8c98-9fea2a39ac6f service glance] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.428570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.429196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.429499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.429921 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] PATH_INFO: `/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a/users/793e2e92c68b4617b2fbfcd57c8ae36b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.450266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=875744a4e5fa459cb3c1934b21d8ff4a, user_id=793e2e92c68b4617b2fbfcd57c8ae36b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.453671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 756/1515] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:54.453815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.454187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.454593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.454979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.473093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 760/1516] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:54 2022] PUT /identity/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a/users/793e2e92c68b4617b2fbfcd57c8ae36b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:54.474079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.480194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:54.481527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.559884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 757/1517] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:54.560822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.562346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.576519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840c1bd-4821-4790-baf3-568ee413adbd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.588995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.590207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.590551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.591058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.592010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.593627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.606067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 761/1518] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:54.606645 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.616380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e09deae-dce6-47fd-935e-61c6ee1eae64 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.621621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.622226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.622537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.622816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.623450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:54.623757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.625357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.639319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 762/1519] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:54.639945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.646334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:54.650753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2259c97b-4b94-4a7d-8100-9d54c2cf9223 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.653239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0cf443-52f2-4305-a6e0-56d003c9b41b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.658737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.659541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.659825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.660101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] PATH_INFO: `/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a/users/793e2e92c68b4617b2fbfcd57c8ae36b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.660336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.660916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.661202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.661480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.662141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=875744a4e5fa459cb3c1934b21d8ff4a, user_id=793e2e92c68b4617b2fbfcd57c8ae36b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.665163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.673316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.675199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7814ac75-555c-43ef-9b8e-34e7500f5886 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.676635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 763/1520] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:54 2022] PUT /identity/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a/users/793e2e92c68b4617b2fbfcd57c8ae36b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:54.677420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.690832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 758/1521] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:54.692319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.692615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.692890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.735294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] MFA Rules not processed for user `793e2e92c68b4617b2fbfcd57c8ae36b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:54.737444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.766742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43fcefd-89a0-47cd-897c-fe51394b0b82 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.771662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.772444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.772733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.773076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.774694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.777722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.784565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:54.810105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 764/1522] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:54.811269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 759/1523] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:54.812050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.824923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.826776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:54.886346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd4a58f-3fd2-4986-8759-fd7a3ea9a68d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.891635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45b019c-eb28-4762-9470-87b6ec200757 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.892626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.893531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.893640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.893910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.894649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.896817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.921107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 765/1524] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:54 2022] POST /identity/v3/users => generated 293 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:54.927572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.941833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4884f1-121b-4230-869f-08d1dbf944ca admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.948671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.949598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.949933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.950214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.950894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.952627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:54.954096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.954853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.955273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.955636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.966514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 766/1525] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:54.970537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:54.982386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38cf03-99bc-47d7-8201-a588fa521661 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:54.989044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:54.989750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:54.990044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:54.990339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:54.991167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:54.993414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.008346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 767/1526] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:11:55.009604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.021103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefdb58b-98a4-49b4-9622-9e484321b5d2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.027634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.028609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.031213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.031505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] PATH_INFO: `/v3/projects/8886ded3870c40e680aeabaa3c656671/users/32fc23f7d72a40b9a4ba4afa2c1290e8/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.040337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.041964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af98cfaf-ca6e-4601-9ba9-6ee8da2aa4b4 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.048527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8886ded3870c40e680aeabaa3c656671, user_id=32fc23f7d72a40b9a4ba4afa2c1290e8, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.052267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.053811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 760/1527] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:54 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:11:55.068847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 768/1528] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:55 2022] PUT /identity/v3/projects/8886ded3870c40e680aeabaa3c656671/users/32fc23f7d72a40b9a4ba4afa2c1290e8/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:55.080087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.080416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.080595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.105182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.107158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:55.115087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] MFA Rules not processed for user `32fc23f7d72a40b9a4ba4afa2c1290e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:55.118226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:55.132716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:55.157987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:55.176965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14274af0-3444-479e-911e-3fa4112b2bb3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.181236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.181842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.182147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.183283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.185902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:55.195277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 761/1529] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:55.199555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.270106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.271521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.272961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517ec096-fd54-49d1-a873-02496d5fd76f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.281981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 769/1530] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:11:55.286650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.287520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.288161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.288516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.289159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.291322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.322896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 762/1531] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:55 2022] POST /identity/v3/projects => generated 387 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:11:55.327743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.354876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6379ce-d9ce-4586-b03b-712a105a046c service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.433992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.434766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.435292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.435586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.436413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.438414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.461871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 770/1532] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:55 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:55.466142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.478876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de341f7e-3fd3-4927-ad79-5fa4789611be admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.484763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.485434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.485719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.486041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.486819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.488648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.500516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 763/1533] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:55.505199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.516461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb4c605-d4c0-4e4e-8716-3dc204dfa64d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.521782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.522516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.522858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.523203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] PATH_INFO: `/v3/projects/89ff7c00102248f5b7791a94a59861e6/users/234c058741d440408e7b557323c54abd/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.539896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ff7c00102248f5b7791a94a59861e6, user_id=234c058741d440408e7b557323c54abd, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.543242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.559497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 771/1534] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:55 2022] PUT /identity/v3/projects/89ff7c00102248f5b7791a94a59861e6/users/234c058741d440408e7b557323c54abd/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:55.564218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.576318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77207429-db3b-412f-9f84-b228d6c47195 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.661093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.661898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.662273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.662678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.663476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.665339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb73bf4d-3042-4264-b874-a1de991299d5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.677636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 764/1535] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:55 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:55.681998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.695884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b23169a-d4bd-461a-b9dd-f85fb3383fdb admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.703280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.704137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.704542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.705059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] PATH_INFO: `/v3/projects/89ff7c00102248f5b7791a94a59861e6/users/234c058741d440408e7b557323c54abd/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.708018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ff7c00102248f5b7791a94a59861e6, user_id=234c058741d440408e7b557323c54abd, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.710282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.723320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 772/1536] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:55 2022] PUT /identity/v3/projects/89ff7c00102248f5b7791a94a59861e6/users/234c058741d440408e7b557323c54abd/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:55.735462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.735790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.736068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.766134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] MFA Rules not processed for user `234c058741d440408e7b557323c54abd`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:55.767425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:55.802002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:55.838399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 765/1537] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:55 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:55.850426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:55.880254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:55.907139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222f3566-a553-4ca9-99c8-8b17a968078b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:55.910783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:55.911386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:55.911668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:55.911978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:55.921231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:55.922729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c05e40-08d6-466b-9ac8-dbe9b75ca34b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:55.932874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 773/1538] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:55 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:55.962073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:56.022989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c043fe07-06b1-4052-82d6-81de7c44f13d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:56.027872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:56.028499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:56.028782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:56.029058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] PATH_INFO: `/v3/users/32fc23f7d72a40b9a4ba4afa2c1290e8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:56.030200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32fc23f7d72a40b9a4ba4afa2c1290e8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:56.032148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:56.096254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32fc23f7d72a40b9a4ba4afa2c1290e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.103471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32fc23f7d72a40b9a4ba4afa2c1290e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.104721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32fc23f7d72a40b9a4ba4afa2c1290e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.112768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32fc23f7d72a40b9a4ba4afa2c1290e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.113480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32fc23f7d72a40b9a4ba4afa2c1290e8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.131259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 766/1539] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:55 2022] DELETE /identity/v3/users/32fc23f7d72a40b9a4ba4afa2c1290e8 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:56.135863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:56.136226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:56.136512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:56.155403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:56.156602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:56.216618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 774/1540] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:56.226352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:56.227741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:56.288059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caa7616-f401-4773-abf7-b890fed4d616 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:56.292523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:56.292986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:56.293265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:56.293539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:56.300083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:56.303955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:56.305948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:56.316963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 767/1541] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:56.751415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:56.752981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:56.794825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414693ec-4d08-4a56-b4f7-2916d8e46960 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:56.801401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:56.802046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:56.802322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:56.802698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] PATH_INFO: `/v3/projects/8886ded3870c40e680aeabaa3c656671` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:56.803849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8886ded3870c40e680aeabaa3c656671)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:56.805738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:56.961885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8886ded3870c40e680aeabaa3c656671', 'request_id': 'req-1f1c7079-6810-4cd1-bcce-acba5e3041a4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.977393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8886ded3870c40e680aeabaa3c656671', 'request_id': 'req-1f1c7079-6810-4cd1-bcce-acba5e3041a4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:56.990152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8886ded3870c40e680aeabaa3c656671', 'request_id': 'req-1f1c7079-6810-4cd1-bcce-acba5e3041a4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.119224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] The token cache is being invalidate because project 8886ded3870c40e680aeabaa3c656671 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:57.119521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.123113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.128681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 775/1542] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:56 2022] DELETE /identity/v3/projects/8886ded3870c40e680aeabaa3c656671 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:57.130312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:57.150074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f9794db-af65-4a40-8bf2-cecb84c5d867 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:57.263134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:57.263772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:57.264049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:57.264318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] PATH_INFO: `/v3/users/234c058741d440408e7b557323c54abd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:57.266856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=234c058741d440408e7b557323c54abd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:57.268914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:57.332160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '234c058741d440408e7b557323c54abd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.340608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '234c058741d440408e7b557323c54abd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.342133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '234c058741d440408e7b557323c54abd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.356681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '234c058741d440408e7b557323c54abd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.357638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-08ac8f6a-a772-4615-9e97-89e882f233ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '234c058741d440408e7b557323c54abd'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.384881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 768/1543] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:57 2022] DELETE /identity/v3/users/234c058741d440408e7b557323c54abd => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:57.775426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:57.781100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:57.818200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c7079-6810-4cd1-bcce-acba5e3041a4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:57.890863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:57.902348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:57.902990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:57.903487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:57.904088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] PATH_INFO: `/v3/projects/89ff7c00102248f5b7791a94a59861e6` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:57.905254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ff7c00102248f5b7791a94a59861e6)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:57.907354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:57.976983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ff7c00102248f5b7791a94a59861e6', 'request_id': 'req-eac5d737-7ec4-47c1-98e1-84054b34430c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:57.993266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ff7c00102248f5b7791a94a59861e6', 'request_id': 'req-eac5d737-7ec4-47c1-98e1-84054b34430c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:58.001625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ff7c00102248f5b7791a94a59861e6', 'request_id': 'req-eac5d737-7ec4-47c1-98e1-84054b34430c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:58.018354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] The token cache is being invalidate because project 89ff7c00102248f5b7791a94a59861e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:11:58.018934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:58.028085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eac5d737-7ec4-47c1-98e1-84054b34430c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:58.030323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 776/1544] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:11:57 2022] DELETE /identity/v3/projects/89ff7c00102248f5b7791a94a59861e6 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:58.034957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.035255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.035890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.057672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:58.059705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:58.156738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 769/1545] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:11:58.164197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.164570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.164925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.188890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:58.190580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:58.200643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 777/1546] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:58.205416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.220365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8bf1-0190-4dd3-b1f3-83df09340748 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.225444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.226016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.226297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.226610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.227274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:11:58.227582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.229661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.249062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:58.256858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 770/1547] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:11:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:11:58.258329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.269412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1557601-35a6-46d5-939c-396de20b09c7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.274365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.275099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.275584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.276526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.277556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.282535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.317856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 778/1548] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:11:58.318944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.329796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a2aa3b-65b3-4e3a-8b6a-0b4efee92da8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.391065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.391675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.391955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.392230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.392859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.394811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.420570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 771/1549] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:58.425303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.437089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33a1526-51e5-4e52-b190-fea03cb54e23 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.441907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.442788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.443073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.443347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.444295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.445911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.463511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 779/1550] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:58.468069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.479689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8646f0ca-a2d1-4dc5-aab2-74d5820c976b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.491050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.491713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.491999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.492275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.492975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.494666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706ce110-529c-411e-8791-89864c25c87a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.507498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 772/1551] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:58.512892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.524135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8396a6b-397a-4bae-aa38-110e396741b5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.531721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.532841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.533321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.533790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] PATH_INFO: `/v3/projects/710c14130ad847c79e5c0facf770d15d/users/6878d54a192545909592d3182c2ef33e/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.552407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] RBAC: Authorizing `identity:create_grant(project_id=710c14130ad847c79e5c0facf770d15d, user_id=6878d54a192545909592d3182c2ef33e, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.555103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.567384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 780/1552] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:58 2022] PUT /identity/v3/projects/710c14130ad847c79e5c0facf770d15d/users/6878d54a192545909592d3182c2ef33e/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:58.579878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.580169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.580449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.616238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] MFA Rules not processed for user `6878d54a192545909592d3182c2ef33e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:58.617472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:58.714713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 773/1553] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:58.716950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.808530 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab725d99-d5fd-42cb-9e81-8a370f33c16f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.814276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.814938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.815220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.815497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.816139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.818082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.854481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 781/1554] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:58.855901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:58.868513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adcaa5e-3c2c-46e4-9f29-151159303b4c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:58.951482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:58.955810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:58.956153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:58.956435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:58.957625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:58.959188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:58.976353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:58.991473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 774/1555] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:11:58 2022] POST /identity/v3/users => generated 300 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:11:58.996644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.004577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:59.015841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b1efe-5342-40a6-889f-e7ad73e459ed admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.021210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.021900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.022643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.022932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.023831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.027408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.043259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 782/1556] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:59.047426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.059179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456c6803-ab82-4542-aaec-13e6731b3c4b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.070175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.071057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.071529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.071860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] PATH_INFO: `/v3/projects/8f660a18189c4e1fbcd84935833547b3/users/e144165f5ee5421d8ded82bf241dfc36/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.091158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f660a18189c4e1fbcd84935833547b3, user_id=e144165f5ee5421d8ded82bf241dfc36, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.093779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.110181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 775/1557] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:59 2022] PUT /identity/v3/projects/8f660a18189c4e1fbcd84935833547b3/users/e144165f5ee5421d8ded82bf241dfc36/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:11:59.114192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.127363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66f8bf4-15d5-49cc-877d-18afbe10a154 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.274010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.276753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:59.326238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.326903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.327187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.327669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.328738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.331338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.355543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 783/1558] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:11:59 2022] GET /identity/v3/roles => generated 1420 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:11:59.359919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.375390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473eb12f-b229-46a8-be01-4cf09a4f505a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.383020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2faf9-5829-4b19-867c-f8b681f414a8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.390108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.390811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.391161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.391471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.391586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.391822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.391914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] PATH_INFO: `/v3/projects/8f660a18189c4e1fbcd84935833547b3/users/e144165f5ee5421d8ded82bf241dfc36/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.392167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] PATH_INFO: `/v3/users/27c7d5a35c8746c09b4edf72b19e5721` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.393331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27c7d5a35c8746c09b4edf72b19e5721)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.393966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f660a18189c4e1fbcd84935833547b3, user_id=e144165f5ee5421d8ded82bf241dfc36, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.395346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.396946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26a250-e03d-42a1-8b2b-38094d6c793b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.408744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 784/1559] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:11:59 2022] PUT /identity/v3/projects/8f660a18189c4e1fbcd84935833547b3/users/e144165f5ee5421d8ded82bf241dfc36/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:59.421814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14277267-cc62-4a29-a4c3-00393940263f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.422093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14277267-cc62-4a29-a4c3-00393940263f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.422513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14277267-cc62-4a29-a4c3-00393940263f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.456078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-14277267-cc62-4a29-a4c3-00393940263f None None] MFA Rules not processed for user `e144165f5ee5421d8ded82bf241dfc36`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:59.457287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14277267-cc62-4a29-a4c3-00393940263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:59.466905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27c7d5a35c8746c09b4edf72b19e5721'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:59.474167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27c7d5a35c8746c09b4edf72b19e5721'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:59.474867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27c7d5a35c8746c09b4edf72b19e5721'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:59.483222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27c7d5a35c8746c09b4edf72b19e5721'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:59.484002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f1283270-76e5-4745-9980-a81930324054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27c7d5a35c8746c09b4edf72b19e5721'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:11:59.506055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 776/1560] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:59 2022] DELETE /identity/v3/users/27c7d5a35c8746c09b4edf72b19e5721 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:11:59.510884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.511418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.511871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.535891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:11:59.537109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:59.564624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 785/1561] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:11:59.580667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14277267-cc62-4a29-a4c3-00393940263f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.582670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14277267-cc62-4a29-a4c3-00393940263f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:59.595366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:59.612848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 777/1562] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:11:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:11:59.630715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:11:59.632329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:11:59.659520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14277267-cc62-4a29-a4c3-00393940263f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.676730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:11:59.734507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.735257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.735613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.736002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.736412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e5a8aa-89ab-46f8-8b61-a518644f27ee None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:11:59.740538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:11:59.741717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:11:59.741717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:11:59.741858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:11:59.755129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.759302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.777767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 778/1563] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:11:59.823739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:11:59.825205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f3b5fe-f308-4761-ab45-7598b14e00df service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:11:59.842476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 786/1564] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:11:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:00.001173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:00.002904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.019042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97990d0d-3ddd-484c-8c0c-f94b75a72a11 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:00.030016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:00.030775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.031181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.031515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] PATH_INFO: `/v3/users/6878d54a192545909592d3182c2ef33e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.032724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6878d54a192545909592d3182c2ef33e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:00.034953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:00.096474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6878d54a192545909592d3182c2ef33e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.106105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6878d54a192545909592d3182c2ef33e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.106955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6878d54a192545909592d3182c2ef33e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.117602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6878d54a192545909592d3182c2ef33e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.118509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6878d54a192545909592d3182c2ef33e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.148326 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 779/1565] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:11:59 2022] DELETE /identity/v3/users/6878d54a192545909592d3182c2ef33e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:00.152971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.153281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.153640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.154282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:00.156149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.176220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:00.177586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.254631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6ae77e-bdfb-47d2-991a-28ffd52e4871 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:00.259383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:00.260059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.260964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.261249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] PATH_INFO: `/v3/projects/e7d8654164f34ca49f73ba503a0caa8f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.262208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 787/1566] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:00.262402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7d8654164f34ca49f73ba503a0caa8f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:00.265061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:00.273183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:00.276322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.285124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:00.329611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7d8654164f34ca49f73ba503a0caa8f', 'request_id': 'req-d3346baa-346d-4c58-9ba7-e5d366b0ab28'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.336307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7d8654164f34ca49f73ba503a0caa8f', 'request_id': 'req-d3346baa-346d-4c58-9ba7-e5d366b0ab28'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.342769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7d8654164f34ca49f73ba503a0caa8f', 'request_id': 'req-d3346baa-346d-4c58-9ba7-e5d366b0ab28'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.350556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873a5e4-7652-4b52-9f60-ef4b1b125258 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:00.357253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] The token cache is being invalidate because project e7d8654164f34ca49f73ba503a0caa8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:00.357615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.358344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.359908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 780/1567] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:00 2022] DELETE /identity/v3/projects/e7d8654164f34ca49f73ba503a0caa8f => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:00.364077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:00.365410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.375305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:00.415331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:00.416188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.417036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.417445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.420710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.461946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3346baa-346d-4c58-9ba7-e5d366b0ab28 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:00.469031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:00.470024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.470709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.471131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] PATH_INFO: `/v3/users/793e2e92c68b4617b2fbfcd57c8ae36b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.472606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=793e2e92c68b4617b2fbfcd57c8ae36b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:00.474762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:00.503285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:00.505029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:00.516652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 788/1568] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:00.534573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '793e2e92c68b4617b2fbfcd57c8ae36b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.544981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '793e2e92c68b4617b2fbfcd57c8ae36b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.546406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '793e2e92c68b4617b2fbfcd57c8ae36b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.560012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '793e2e92c68b4617b2fbfcd57c8ae36b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.560702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c096639-8b9e-4214-8ffb-36c4a2dd1e5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '793e2e92c68b4617b2fbfcd57c8ae36b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.583927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 781/1569] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:00 2022] DELETE /identity/v3/users/793e2e92c68b4617b2fbfcd57c8ae36b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:00.668037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:00.669491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.696460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0253cc56-b97b-4a7a-b979-8fe5c35c5ac5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:00.763955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28790534-0320-4147-8775-275bdad82dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:00.764729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28790534-0320-4147-8775-275bdad82dba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.765061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28790534-0320-4147-8775-275bdad82dba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.765400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28790534-0320-4147-8775-275bdad82dba admin admin] PATH_INFO: `/v3/users/6246d4721a3546e88d118daf8e3fa78e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.766628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28790534-0320-4147-8775-275bdad82dba admin admin] RBAC: Authorizing `identity:delete_user(user_id=6246d4721a3546e88d118daf8e3fa78e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:00.768618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28790534-0320-4147-8775-275bdad82dba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:00.867459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6246d4721a3546e88d118daf8e3fa78e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.877562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6246d4721a3546e88d118daf8e3fa78e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.891339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6246d4721a3546e88d118daf8e3fa78e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.892125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6246d4721a3546e88d118daf8e3fa78e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.892863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6246d4721a3546e88d118daf8e3fa78e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:00.907160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:00.920854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 789/1570] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:00 2022] DELETE /identity/v3/users/6246d4721a3546e88d118daf8e3fa78e => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:00.926595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:00.927170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:00.927652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:00.954626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:00.956563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:00.985516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:01.059240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.064056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.083660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 782/1571] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:01.087907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.090407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.103525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1eb3594-9677-423b-adb7-12020f8cca8f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.110514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.111778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.112164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.112881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] PATH_INFO: `/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.115969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=875744a4e5fa459cb3c1934b21d8ff4a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.118235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.214964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '875744a4e5fa459cb3c1934b21d8ff4a', 'request_id': 'req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.220969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28790534-0320-4147-8775-275bdad82dba admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.226601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '875744a4e5fa459cb3c1934b21d8ff4a', 'request_id': 'req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.241920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '875744a4e5fa459cb3c1934b21d8ff4a', 'request_id': 'req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.265889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] The token cache is being invalidate because project 875744a4e5fa459cb3c1934b21d8ff4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:01.266188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.267093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.269754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 783/1572] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:01 2022] DELETE /identity/v3/projects/875744a4e5fa459cb3c1934b21d8ff4a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:01.270914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.273318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.313459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.314223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.314538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.314817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] PATH_INFO: `/v3/projects/710c14130ad847c79e5c0facf770d15d` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.315983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=710c14130ad847c79e5c0facf770d15d)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.318534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.397893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '710c14130ad847c79e5c0facf770d15d', 'request_id': 'req-ee582deb-2a9e-47d0-a13b-cba62405eb9f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.405447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '710c14130ad847c79e5c0facf770d15d', 'request_id': 'req-ee582deb-2a9e-47d0-a13b-cba62405eb9f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.408438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1602ae3-0994-4fae-acfd-adb8f2f43c6a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.417548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '710c14130ad847c79e5c0facf770d15d', 'request_id': 'req-ee582deb-2a9e-47d0-a13b-cba62405eb9f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.439811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] The token cache is being invalidate because project 710c14130ad847c79e5c0facf770d15d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:01.440198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.445387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ee582deb-2a9e-47d0-a13b-cba62405eb9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.445387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 790/1573] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:01 2022] DELETE /identity/v3/projects/710c14130ad847c79e5c0facf770d15d => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:01.445387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.445721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.445806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.480050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:01.482452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.490054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.490828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.491419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.491711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.493741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.529143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:01.535834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:01.580936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 791/1574] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:01.587481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.587481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.591590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.593210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f701dac-59be-42f1-9d77-e5f73fd47043 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.598371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4accaa-bbab-4acb-8714-6d9a525b7139 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.603160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.603790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 784/1575] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:01.604300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.604651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.605000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] PATH_INFO: `/v3/users/e144165f5ee5421d8ded82bf241dfc36` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.605098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.605535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.605821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.606498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e144165f5ee5421d8ded82bf241dfc36)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.608463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.627385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:01.628613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.638848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 785/1576] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:01.644045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.655813 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462ad5b-55c1-4276-a7d9-64b93b7db59b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.661280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.662464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.662783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.663060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.664860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:01.665170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.669285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.669485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e144165f5ee5421d8ded82bf241dfc36'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.679208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e144165f5ee5421d8ded82bf241dfc36'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.690547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 786/1577] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:01.693578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.693730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e144165f5ee5421d8ded82bf241dfc36'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.694357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e144165f5ee5421d8ded82bf241dfc36'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.698744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e144165f5ee5421d8ded82bf241dfc36'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:01.707477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323242a1-4b03-4a99-99af-31850bb3fe11 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.714205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.714848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.715129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.715401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.716061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.716458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 792/1578] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:01 2022] DELETE /identity/v3/users/e144165f5ee5421d8ded82bf241dfc36 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:01.718011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:01.751465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 787/1579] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:01 2022] POST /identity/v3/projects => generated 393 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:01.752614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:01.754584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:01.832758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802f595d-bae9-4a87-8c6e-e7f4b247f2e7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:01.940887 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:01.963199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:01.964243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:01.964780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:01.965285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:01.966168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:01.968918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.006106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 793/1580] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:01 2022] POST /identity/v3/users => generated 297 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:02.007438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.028526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae10beab-9d55-4e1c-8011-935dfc452f01 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.035240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.038909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.040176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.040658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.041917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.044168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.065096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.077849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 788/1581] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:02.079590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8b8322-841e-4703-838e-4941db44dd6b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.108520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.109670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.110649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.111136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.112015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.113886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd83ee6-cd17-4e2f-94af-d2baa5638f1d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.127674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 794/1582] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:02.132233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.146671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad70b8bb-fc7f-4cde-8f4e-947aa478317f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.157154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.158282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.158775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.159204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] PATH_INFO: `/v3/projects/fabbd15be617406e937f93acbefb305e/users/198469edb0ed44259f8b3cfb3683ab16/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.187467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fabbd15be617406e937f93acbefb305e, user_id=198469edb0ed44259f8b3cfb3683ab16, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.190580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.216616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 789/1583] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:02 2022] PUT /identity/v3/projects/fabbd15be617406e937f93acbefb305e/users/198469edb0ed44259f8b3cfb3683ab16/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:02.228416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.228923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.229317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.270066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] MFA Rules not processed for user `198469edb0ed44259f8b3cfb3683ab16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:02.271679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.347017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 795/1584] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:02 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:02.352224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.458238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad146ff-5d07-4d4d-af6f-ff633cded410 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.467889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.468832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.469185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.469707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.470351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.472495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.501335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.503110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.518079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 790/1585] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:02 2022] POST /identity/v3/projects => generated 391 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:02.523966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.562367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d49539-20d4-4bb4-baec-9e0c55ab2733 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.569704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:02.640765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06843ca1-4d20-4ca2-8bc3-ddd4f70e1363 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.646227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.646883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.647167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.647445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.648306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.650870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.667122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.667777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.668059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.668333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] PATH_INFO: `/v3/projects/8f660a18189c4e1fbcd84935833547b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.669861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f660a18189c4e1fbcd84935833547b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.673831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.689640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 791/1586] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:02 2022] POST /identity/v3/users => generated 302 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:02.690571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.692690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.710552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba196d03-f031-46d6-9325-7d968988704b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.717071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.717830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.718251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.718632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] PATH_INFO: `/v3/projects/dc69877a9a0c494886d525c40e6bcf06` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.720387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc69877a9a0c494886d525c40e6bcf06)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.723485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.769664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f660a18189c4e1fbcd84935833547b3', 'request_id': 'req-8bb01f0a-4245-4857-8682-f9cf35981e68'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.781680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f660a18189c4e1fbcd84935833547b3', 'request_id': 'req-8bb01f0a-4245-4857-8682-f9cf35981e68'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.792919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:02.794688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f660a18189c4e1fbcd84935833547b3', 'request_id': 'req-8bb01f0a-4245-4857-8682-f9cf35981e68'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.817351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] The token cache is being invalidate because project 8f660a18189c4e1fbcd84935833547b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:02.817784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.818532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.820059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 796/1587] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:02 2022] DELETE /identity/v3/projects/8f660a18189c4e1fbcd84935833547b3 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:02.824774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.826191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc69877a9a0c494886d525c40e6bcf06', 'request_id': 'req-f4caa062-2dfe-45da-b776-563b02646d0d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.826337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.834896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc69877a9a0c494886d525c40e6bcf06', 'request_id': 'req-f4caa062-2dfe-45da-b776-563b02646d0d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.844019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc69877a9a0c494886d525c40e6bcf06', 'request_id': 'req-f4caa062-2dfe-45da-b776-563b02646d0d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.860302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] The token cache is being invalidate because project dc69877a9a0c494886d525c40e6bcf06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:02.860677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.861272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f4caa062-2dfe-45da-b776-563b02646d0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:02.863107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 792/1588] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:02 2022] DELETE /identity/v3/projects/dc69877a9a0c494886d525c40e6bcf06 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:02.867726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.868169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.868577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.893415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:02.894674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.904253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb01f0a-4245-4857-8682-f9cf35981e68 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:02.974883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 793/1589] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:02.976095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:02.980691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45407397-708a-4232-9151-d6535f447831 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:02.981990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:02.982806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407397-708a-4232-9151-d6535f447831 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:02.983303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407397-708a-4232-9151-d6535f447831 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:02.983778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45407397-708a-4232-9151-d6535f447831 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:02.984637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45407397-708a-4232-9151-d6535f447831 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:02.986732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45407397-708a-4232-9151-d6535f447831 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:02.996175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c9a05d-29f8-4db7-aa75-36963d211317 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.003099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 797/1590] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:02 2022] GET /identity/v3/roles => generated 1420 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:03.004090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d02-149b-4824-9183-69a18062b067 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.004378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d02-149b-4824-9183-69a18062b067 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.004654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae07d02-149b-4824-9183-69a18062b067 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.012230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.013145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.014229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.014933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] PATH_INFO: `/v3/users/619c3e32fe4f40ff949eba15c994bf34` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.017249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] RBAC: Authorizing `identity:delete_user(user_id=619c3e32fe4f40ff949eba15c994bf34)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.024020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.027652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dae07d02-149b-4824-9183-69a18062b067 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:03.029118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dae07d02-149b-4824-9183-69a18062b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.037404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 798/1591] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:03.038479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae07d02-149b-4824-9183-69a18062b067 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.039714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dae07d02-149b-4824-9183-69a18062b067 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.053947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae07d02-149b-4824-9183-69a18062b067 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.060782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:INSERT=4,keystone:DELETE=11 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:03.065592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.066328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.066849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.067332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] PATH_INFO: `/v3/projects/670c32a6a94642e2ae6f21c887239fd0/users/3081be2372194380ae10827c8cfcf514/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.084126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c32a6a94642e2ae6f21c887239fd0, user_id=3081be2372194380ae10827c8cfcf514, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.087448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.096577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619c3e32fe4f40ff949eba15c994bf34'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.101920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 799/1592] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:03 2022] PUT /identity/v3/projects/670c32a6a94642e2ae6f21c887239fd0/users/3081be2372194380ae10827c8cfcf514/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:03.102714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.108731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619c3e32fe4f40ff949eba15c994bf34'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.109772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619c3e32fe4f40ff949eba15c994bf34'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.124830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619c3e32fe4f40ff949eba15c994bf34'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.125792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619c3e32fe4f40ff949eba15c994bf34'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.149143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 794/1593] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:02 2022] DELETE /identity/v3/users/619c3e32fe4f40ff949eba15c994bf34 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:03.149780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.152011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.189613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d32e7-c677-4c96-9238-97b13ebc78e6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.196786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:03.247544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb47f37-b28f-49d4-89e2-44216c16b686 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.257713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.258345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.258661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.258943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.259620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.263313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.276850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 795/1594] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:03.281643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.294018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026f1cc3-0083-4035-807e-742a6ecb27e4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.298842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.299759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.300040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.300327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] PATH_INFO: `/v3/projects/670c32a6a94642e2ae6f21c887239fd0/users/3081be2372194380ae10827c8cfcf514/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.302566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] RBAC: Authorizing `identity:create_grant(project_id=670c32a6a94642e2ae6f21c887239fd0, user_id=3081be2372194380ae10827c8cfcf514, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.304040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.305245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47a80d2-d965-451e-8646-1a96575a0768 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.305632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.307001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.311058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.311641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:03.311958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.313607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be39c77-02fa-4c61-9311-274288393c8c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.322021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 796/1595] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:03 2022] PUT /identity/v3/projects/670c32a6a94642e2ae6f21c887239fd0/users/3081be2372194380ae10827c8cfcf514/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:03.324399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.324695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.325165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.341424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 800/1596] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:03.343077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fb24-eccb-407b-9f96-9e248717365e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.343369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fb24-eccb-407b-9f96-9e248717365e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.343646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e857fb24-eccb-407b-9f96-9e248717365e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.345471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 797/1597] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:12:03 2022] GET /identity => generated 270 bytes in 22 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:12:03.346525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.351592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.378039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e857fb24-eccb-407b-9f96-9e248717365e None None] MFA Rules not processed for user `3081be2372194380ae10827c8cfcf514`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:03.379415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e857fb24-eccb-407b-9f96-9e248717365e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.459127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67abc826-7797-4b20-8e44-32f702b5e59c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.464928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.465544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.465828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.466104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.466775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.468961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.472089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 801/1598] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:03.473443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.473955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.474238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.503247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:03.505162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.508727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 798/1599] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:03.513180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.514897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.530665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:03.589360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 802/1600] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:03.590327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.592773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501c55e6-7fae-40b4-913a-5fa4faf529be admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.596922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.598087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.598372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.598675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.612806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:03.659701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b31b9d-7d93-4069-9f97-9e9cdd2ad9da None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.664349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.664945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.665222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.665491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.667335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.669245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.677128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.685223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.689883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 799/1601] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:03.690837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.692182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.711231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd0ac2-9d48-4967-9b0f-ef16b6993345 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.712014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 803/1602] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/users => generated 293 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:03.713532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.715656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.721316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.722095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.722382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.722697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] PATH_INFO: `/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.723816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ebd0f2a123a42f4bc6809f837510eb2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.725986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.730676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b194872-2d91-41b2-9ab1-845aa33771b8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.736492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.737095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.737373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.737644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.747552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.749275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.761107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 804/1603] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:03.762163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.775243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacc8be7-73bf-4a2e-9c32-919482e5d39e service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.792607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.793405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.793791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.794130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.795134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.800755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.808997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ebd0f2a123a42f4bc6809f837510eb2', 'request_id': 'req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.817771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 805/1604] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:03.817771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.817972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.818114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ebd0f2a123a42f4bc6809f837510eb2', 'request_id': 'req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.825575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ebd0f2a123a42f4bc6809f837510eb2', 'request_id': 'req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.836093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435b0a87-6d94-4734-bd86-5ab2421bef24 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.849443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] The token cache is being invalidate because project 2ebd0f2a123a42f4bc6809f837510eb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:03.850544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.851506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:03.855632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 800/1605] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:03 2022] DELETE /identity/v3/projects/2ebd0f2a123a42f4bc6809f837510eb2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:03.856853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:03.863642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.907408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.908022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.908297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.908572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.910589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:03.923337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:03.956352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf38355e-fdf1-47de-9a59-4a8cf515c1a9 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:03.961540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:03.962174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.962479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.963854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.964743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.966567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258dd66c-6ae9-4fb0-8333-1e64ca448e33 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:03.979872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 801/1606] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:03.981118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb2b4-e2c7-4103-96a5-8a7ed3d3d369 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:03.981398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb2b4-e2c7-4103-96a5-8a7ed3d3d369 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:03.981676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926bb2b4-e2c7-4103-96a5-8a7ed3d3d369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:03.985782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:03.987866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.000449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 806/1607] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:04.000449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.012388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e933bd-2aed-4967-b713-21b56ea501f7 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.016626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-926bb2b4-e2c7-4103-96a5-8a7ed3d3d369 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:04.017374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.017861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-926bb2b4-e2c7-4103-96a5-8a7ed3d3d369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:04.018266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.018644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.018984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] PATH_INFO: `/v3/projects/601dda0a38fd4a3099a5d544384ad45b/users/09af24f8664d4b909d66192872fc01b5/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.030921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 802/1608] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:04.041040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.041329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.041647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.051209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=601dda0a38fd4a3099a5d544384ad45b, user_id=09af24f8664d4b909d66192872fc01b5, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.055599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504beb8-4917-4094-bf43-ca55f695ae0c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.069186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:04.070819 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:04.084255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 807/1609] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:03 2022] PUT /identity/v3/projects/601dda0a38fd4a3099a5d544384ad45b/users/09af24f8664d4b909d66192872fc01b5/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:04.100415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.100962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.101412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.157933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 803/1610] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:04.159096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.159347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] MFA Rules not processed for user `09af24f8664d4b909d66192872fc01b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:04.160755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:04.169561 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:04.179768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d462a483-c8c1-4887-a29b-b5c55de6b37c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.188358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.189226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.189783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.190094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.191675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:04.191675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.194191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.224929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 804/1611] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:04.227716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.248640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c922e0eb-632a-4d26-9a88-6d3525188493 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.263797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.264899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.265563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.266150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.267058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.270016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.303114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 808/1612] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:04.307447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.329982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 805/1613] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:04.331965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5ac01-739f-477e-aed3-1355b837474f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.334894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.444795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.445424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.445707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.446327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.446938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.450277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.460343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b29c1b-e756-4134-b2a3-458e47e98123 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.468438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.469374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.469844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.470224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.471116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.480406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.501104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 809/1614] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/projects => generated 387 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:04.506177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.508039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 806/1615] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/users => generated 290 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:04.514283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.524862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f000f40-c793-41da-b57a-e3b105542180 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.553514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:04.589960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dfb235-1cb5-4fff-afa1-5179c095c583 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.596681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.597481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.597861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.598146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.598855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.600863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.618668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.620504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.620853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.621229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.621877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.623566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.626115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 810/1616] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/users => generated 300 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:04.629996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.633794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 807/1617] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:04.638143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.640401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbea360-b6ad-4a40-bb54-83e62045233c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.644963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.645558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.645841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.646114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.646780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.648169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ecd9d3-7d72-4779-a1ae-dc0cd0a11663 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.648384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.652885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.653474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.653760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.654035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.654697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.656347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.659673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 811/1618] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:04.663711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.674510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 808/1619] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:04.674658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0c9f5-931e-46fa-a27d-95d6010a2ca4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.675531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.685224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.685224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.685224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.685224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] PATH_INFO: `/v3/projects/04acbc733e6f4c6786defbcabe80165e/users/a4e5418fce35488a859186dbde9f8e76/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.685754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:04.686690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51453688-24ef-404a-8e1a-55c2cf8efa71 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.693657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.694289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.694612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.694889 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] PATH_INFO: `/v3/projects/12b73748d6404120b1606bd318c32bf4/users/bcfa077ca70841eb86137bd39eb7a281/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.703558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04acbc733e6f4c6786defbcabe80165e, user_id=a4e5418fce35488a859186dbde9f8e76, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.706762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.712500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b73748d6404120b1606bd318c32bf4, user_id=bcfa077ca70841eb86137bd39eb7a281, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.716244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccc3d8-6f19-4796-8fc0-c95ee7e0aebb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.722962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 812/1620] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:04 2022] PUT /identity/v3/projects/04acbc733e6f4c6786defbcabe80165e/users/a4e5418fce35488a859186dbde9f8e76/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:04.727012 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.733005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 809/1621] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:04 2022] PUT /identity/v3/projects/12b73748d6404120b1606bd318c32bf4/users/bcfa077ca70841eb86137bd39eb7a281/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:04.744091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.744380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.744661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.774941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] MFA Rules not processed for user `bcfa077ca70841eb86137bd39eb7a281`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:04.777099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:04.800796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8137f3-c913-4997-aa09-524fcf5fbfa1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.845034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 810/1622] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:04.849773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.866031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.866782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.867169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.867531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.868182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.870741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.881418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 813/1623] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:04.886033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:04.896273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b1f41-5ce5-48f5-9eca-3f422e7951b0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.902136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:04.903241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.904022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.904773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] PATH_INFO: `/v3/projects/04acbc733e6f4c6786defbcabe80165e/users/a4e5418fce35488a859186dbde9f8e76/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.906995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04acbc733e6f4c6786defbcabe80165e, user_id=a4e5418fce35488a859186dbde9f8e76, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:04.909818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a33fa4-d1d8-457f-b342-825c667f44b9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:04.919977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2312edfe-f2d4-4ed9-bb5d-fcf5179d3b1d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:04.932401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 814/1624] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:04 2022] PUT /identity/v3/projects/04acbc733e6f4c6786defbcabe80165e/users/a4e5418fce35488a859186dbde9f8e76/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:04.935677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:04.936146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:04.936322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:04.958675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:04.984523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] MFA Rules not processed for user `a4e5418fce35488a859186dbde9f8e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:04.986028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:05.031916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.033492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.034134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.034684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.036186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.042601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.090324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 811/1625] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/projects => generated 377 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:05.097834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:05.107915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:05.108971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 815/1626] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:04 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:05.120129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:05.121810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:05.191773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b5a13-c219-4c49-ac27-fedb88eaff12 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:05.205390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.208126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.208365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.208657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.209315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.212944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.239774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5631a35-d82e-435c-a607-92d34ddd747e None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:05.280423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 812/1627] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:05 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:05.282524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:05.311982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03021aab-bb35-4c54-bd90-9e8dc1101bd3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:05.314493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.315209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.315731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.316101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.327731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.329139 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.329428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.329770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.332270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.333952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.364618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 813/1628] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:05.365591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:05.389319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b0721e-6801-4bd8-9983-7c56a1bd1bec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:05.403151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.404083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.404480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.404756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.405627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.423650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3b9222-2023-4e02-9d2e-687b87e9fcfc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.424099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.424099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.439109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 816/1629] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:05 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:12:05.444896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 814/1630] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:05.450128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:05.467430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ec95b8-9dc3-4858-9fcc-204a7a41d819 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:05.483467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:05.484166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.484453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.484721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] PATH_INFO: `/v3/projects/6f97908764f441628d46a24049cbe7ad/users/0fdc06360aa14f60bf1a63c3a8a39619/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.524347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f97908764f441628d46a24049cbe7ad, user_id=0fdc06360aa14f60bf1a63c3a8a39619, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:05.529094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:05.566632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 817/1631] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:05 2022] PUT /identity/v3/projects/6f97908764f441628d46a24049cbe7ad/users/0fdc06360aa14f60bf1a63c3a8a39619/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:05.573158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:05.573750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:05.573750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:05.705428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] MFA Rules not processed for user `0fdc06360aa14f60bf1a63c3a8a39619`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:05.710835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:05.915240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 815/1632] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:05.919758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.004554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:06.040195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28123235-73dc-4a98-8a02-18a602fc89b5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.050556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.051421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.051705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.051974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.052680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.055079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.105863 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 818/1633] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:05 2022] POST /identity/v3/projects => generated 377 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:06.121839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.142662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eab31cc-1220-4cdf-83f1-9a8853b4bac1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.149365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:06.255068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.255985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.256419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.256845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.258374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.262070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.305414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 816/1634] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:06 2022] POST /identity/v3/users => generated 295 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:06.309818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.324619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f63e036-b7c9-4c5e-a307-048c0ff3e698 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.330740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.332514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.332808 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.333233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.333872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.336344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.351908 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 819/1635] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:06.356986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.370604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2534b7-1b1a-4079-bc74-ab1f08a4f14e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.398362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.399280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.399772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.399908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] PATH_INFO: `/v3/projects/e1ece61fbf7541c8bae1f73443a799cb/users/b5dfc4f5392c482e8c284bd55da80d93/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.422105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.430093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ece61fbf7541c8bae1f73443a799cb, user_id=b5dfc4f5392c482e8c284bd55da80d93, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.433799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.435000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e65b25-3bf8-4829-a125-eabd6b6b7a44 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.441056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.441730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.442014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.442306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] PATH_INFO: `/v3/users/09af24f8664d4b909d66192872fc01b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.443578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09af24f8664d4b909d66192872fc01b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.445562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.471127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 817/1636] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:06 2022] PUT /identity/v3/projects/e1ece61fbf7541c8bae1f73443a799cb/users/b5dfc4f5392c482e8c284bd55da80d93/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:06.475285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.504163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09af24f8664d4b909d66192872fc01b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:06.511956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09af24f8664d4b909d66192872fc01b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:06.520277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09af24f8664d4b909d66192872fc01b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:06.520949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09af24f8664d4b909d66192872fc01b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:06.521578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c8df145b-4081-42b3-82f6-c70e68b168d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09af24f8664d4b909d66192872fc01b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:06.547747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 820/1637] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:06 2022] DELETE /identity/v3/users/09af24f8664d4b909d66192872fc01b5 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:06.552529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.552818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.553094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.567483 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f78186-65a1-4c7f-984c-31824379fd5f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.582915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:06.584128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:06.644464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.645121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.645478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.645839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.646477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.648106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.653481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 821/1638] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:06.669033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.669033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:06.669210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 818/1639] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:06 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:06.671033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.673065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:06.689193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf7962-6921-4d0d-9cd1-1e14fc02e0a7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.694636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.695415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.695762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.696106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] PATH_INFO: `/v3/projects/e1ece61fbf7541c8bae1f73443a799cb/users/b5dfc4f5392c482e8c284bd55da80d93/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.698877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1ece61fbf7541c8bae1f73443a799cb, user_id=b5dfc4f5392c482e8c284bd55da80d93, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.701695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8f38a8-8edc-4383-84ab-d8e41c494426 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.713603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 819/1640] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:06 2022] PUT /identity/v3/projects/e1ece61fbf7541c8bae1f73443a799cb/users/b5dfc4f5392c482e8c284bd55da80d93/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:06.714164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:06.725804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.726093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.726380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.735152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:06.749991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aae6c94-67f6-4d7c-bcff-d2648b9c6714 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.763799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] MFA Rules not processed for user `b5dfc4f5392c482e8c284bd55da80d93`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:06.765626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:06.823470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.824553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.825538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.826209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.861935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 820/1641] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:06 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:06.877838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:06.879541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:06.947389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.948881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:06.960893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 822/1642] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:12:06.971301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ed28f1-4ced-4e10-849a-a5851047550f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:06.974928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:06.975648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:06.975930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:06.976203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:06.986522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:06.987942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:07.000016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 821/1643] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:07.054727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:07.056827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:07.070230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1942438b-9420-43fd-b8a6-6ee29adb9d26 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:07.076316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:07.076947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:07.077240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:07.077518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:07.079738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:07.190591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:07.192366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:07.210590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 823/1644] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:07 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:07.376209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:07.377664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:07.394208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e90463-6e5d-4bce-988c-a8a1f75cccd8 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:07.401257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:07.401892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:07.402177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:07.402482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] PATH_INFO: `/v3/projects/601dda0a38fd4a3099a5d544384ad45b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:07.403648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=601dda0a38fd4a3099a5d544384ad45b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:07.405532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:07.460767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '601dda0a38fd4a3099a5d544384ad45b', 'request_id': 'req-3b1d5422-9f3c-4b80-83c2-13943d7fd395'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.470134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '601dda0a38fd4a3099a5d544384ad45b', 'request_id': 'req-3b1d5422-9f3c-4b80-83c2-13943d7fd395'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.481411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '601dda0a38fd4a3099a5d544384ad45b', 'request_id': 'req-3b1d5422-9f3c-4b80-83c2-13943d7fd395'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.499325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] The token cache is being invalidate because project 601dda0a38fd4a3099a5d544384ad45b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:07.499754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.500643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.502472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 822/1645] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:07 2022] DELETE /identity/v3/projects/601dda0a38fd4a3099a5d544384ad45b => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:07.506680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:07.523786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25157ace-8889-4ad0-9da7-f267828c583b service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:07.574328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:07.610471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:07.612007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:07.612478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:07.612957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] PATH_INFO: `/v3/users/a4e5418fce35488a859186dbde9f8e76` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:07.614832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4e5418fce35488a859186dbde9f8e76)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:07.617776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:07.690575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4e5418fce35488a859186dbde9f8e76'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.699708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4e5418fce35488a859186dbde9f8e76'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.709263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4e5418fce35488a859186dbde9f8e76'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.710209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4e5418fce35488a859186dbde9f8e76'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.711245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4e5418fce35488a859186dbde9f8e76'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:07.737815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 824/1646] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:07 2022] DELETE /identity/v3/users/a4e5418fce35488a859186dbde9f8e76 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:08.019105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.020710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.046720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1d5422-9f3c-4b80-83c2-13943d7fd395 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.055900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:08.114191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.114862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.115144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.115423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] PATH_INFO: `/v3/projects/04acbc733e6f4c6786defbcabe80165e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.116535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=04acbc733e6f4c6786defbcabe80165e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.118426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.157871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.159369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.168514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04acbc733e6f4c6786defbcabe80165e', 'request_id': 'req-43c18a38-2834-49cb-8a7c-449c6f7beb9d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.173926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a38417-fad6-4620-9d82-213b836b883c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.175214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04acbc733e6f4c6786defbcabe80165e', 'request_id': 'req-43c18a38-2834-49cb-8a7c-449c6f7beb9d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.181365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04acbc733e6f4c6786defbcabe80165e', 'request_id': 'req-43c18a38-2834-49cb-8a7c-449c6f7beb9d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.194597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] The token cache is being invalidate because project 04acbc733e6f4c6786defbcabe80165e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:08.194904 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.195532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-43c18a38-2834-49cb-8a7c-449c6f7beb9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.197592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 823/1647] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:08 2022] DELETE /identity/v3/projects/04acbc733e6f4c6786defbcabe80165e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:08.202696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15d68-6bd3-4684-924d-ce0082c503d2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.203015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15d68-6bd3-4684-924d-ce0082c503d2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.203375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f15d68-6bd3-4684-924d-ce0082c503d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.224875 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-10f15d68-6bd3-4684-924d-ce0082c503d2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:08.226088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10f15d68-6bd3-4684-924d-ce0082c503d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.235228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.235854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.236132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.236405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] PATH_INFO: `/v3/users/198469edb0ed44259f8b3cfb3683ab16` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.237535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] RBAC: Authorizing `identity:delete_user(user_id=198469edb0ed44259f8b3cfb3683ab16)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.239462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.285594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 824/1648] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:08.286607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '198469edb0ed44259f8b3cfb3683ab16'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.293161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:08.294006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.294363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.294596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.295314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '198469edb0ed44259f8b3cfb3683ab16'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.304224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '198469edb0ed44259f8b3cfb3683ab16'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.305734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '198469edb0ed44259f8b3cfb3683ab16'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.307213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-299e73f4-c4c2-446a-be1e-da90fa2e975e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '198469edb0ed44259f8b3cfb3683ab16'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:08.315788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:08.316988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.325341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 825/1650] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:08 2022] DELETE /identity/v3/users/198469edb0ed44259f8b3cfb3683ab16 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:08.328292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 825/1650] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:08.329584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.329719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.329900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.330176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.338781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ebe4cf-9dbd-4f18-a187-68fca6ec0ad0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.349194 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:08.350383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.376771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:08.407779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.408509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.408966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.409246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.410268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:08.410268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.412496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.424715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 826/1651] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:08.432916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 826/1652] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:12:08.436369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.437019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.437851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.451734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9488fb18-b744-4b4e-9e15-43437796a918 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.459570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.460518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.460802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.461075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.461735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.464814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.500761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636266bc-0b09-4469-b2a0-a9fbbe1a6bc1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.503167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 827/1653] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:08.507546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.508573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.509368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.509807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.510100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.523230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.524671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.535884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 827/1654] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:08.583801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea45fc0-c9fe-437c-9403-abf52c353387 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.589614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.590279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.590544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.590872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.591932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.593592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.621964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 828/1655] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/users => generated 291 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:08.626412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.638766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae47d1c-9585-4a13-be5a-bd21883bfbaa service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.643913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.644489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.644763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.645033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.645828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.647545 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.676856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 828/1656] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:08.680869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.692595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33be69c9-ec3d-4654-bd5b-a04976645747 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.699125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.700167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.700662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.700943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.701652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.703327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91607487-9f6e-4a64-88b1-649e9319408f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.719470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 829/1657] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:08.721638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:08.733557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88515d-1abc-4df9-bb34-868f10205e94 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:08.739763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:08.740712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.740955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.741233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] PATH_INFO: `/v3/projects/97d496cc0636403784a6fd78111a97f8/users/e3da95d1ba214ed99eaf5f17fc3a39ee/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.761590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d496cc0636403784a6fd78111a97f8, user_id=e3da95d1ba214ed99eaf5f17fc3a39ee, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:08.764604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:08.790704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 829/1658] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:08 2022] PUT /identity/v3/projects/97d496cc0636403784a6fd78111a97f8/users/e3da95d1ba214ed99eaf5f17fc3a39ee/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:08.799080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:08.799376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:08.799977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:08.868120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] MFA Rules not processed for user `e3da95d1ba214ed99eaf5f17fc3a39ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:08.870303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:08.958464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 830/1659] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:08.963823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.048202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:09.090587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2507446a-9e1f-457c-b048-e851c2b6278b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.104509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.105144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.127347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.127875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.128748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.152232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.153893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.171024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.190008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2013cd7c-5d1c-44b4-98a3-1dc565a9321b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.200329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.200984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.201278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.201539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] PATH_INFO: `/v3/projects/fabbd15be617406e937f93acbefb305e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.206601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fabbd15be617406e937f93acbefb305e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.208708 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.227514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:09.267735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 830/1660] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:08 2022] POST /identity/v3/projects => generated 379 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:09.268585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.296187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fabbd15be617406e937f93acbefb305e', 'request_id': 'req-d5d68eb1-7a76-40d6-85f2-71239fd028c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.307243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fabbd15be617406e937f93acbefb305e', 'request_id': 'req-d5d68eb1-7a76-40d6-85f2-71239fd028c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.317548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fabbd15be617406e937f93acbefb305e', 'request_id': 'req-d5d68eb1-7a76-40d6-85f2-71239fd028c0'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.343431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] The token cache is being invalidate because project fabbd15be617406e937f93acbefb305e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:09.343724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.345691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.347591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 831/1661] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:09 2022] DELETE /identity/v3/projects/fabbd15be617406e937f93acbefb305e => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:09.351542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.354292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.355987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2ea00-9cb8-40a4-8082-89653877f896 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.428548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.429085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.429636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.429931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.430622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.432646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d68eb1-7a76-40d6-85f2-71239fd028c0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.433334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.438158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.438836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.439277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.439559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] PATH_INFO: `/v3/users/3081be2372194380ae10827c8cfcf514` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.440798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] RBAC: Authorizing `identity:delete_user(user_id=3081be2372194380ae10827c8cfcf514)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.443468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.458478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 831/1662] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:09 2022] POST /identity/v3/users => generated 296 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:09.459837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.461450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.473684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36beb04f-2b1c-43ef-b189-a790c3f7db43 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.479624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.480331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.480614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.480886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.481527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.483213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.493939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 832/1663] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:09.497803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3081be2372194380ae10827c8cfcf514'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.498356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.505404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3081be2372194380ae10827c8cfcf514'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.506329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3081be2372194380ae10827c8cfcf514'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.507877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d435b7c8-874f-4e21-859b-65d0d255cbc1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.514640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.515016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3081be2372194380ae10827c8cfcf514'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.515460 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.515615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3081be2372194380ae10827c8cfcf514'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:09.515810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.516137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] PATH_INFO: `/v3/projects/8a8a976443844a2e91f3f6fe212ac109/users/2645a999f3064c9bbda415619ba766e1/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.533092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8a976443844a2e91f3f6fe212ac109, user_id=2645a999f3064c9bbda415619ba766e1, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.535727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 832/1664] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:09 2022] DELETE /identity/v3/users/3081be2372194380ae10827c8cfcf514 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:09.535847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.548797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 833/1665] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:09 2022] PUT /identity/v3/projects/8a8a976443844a2e91f3f6fe212ac109/users/2645a999f3064c9bbda415619ba766e1/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:09.552719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.554214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.579581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:09.628430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f305ee19-621c-4ef6-9594-a6a7712f394a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.705022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.706104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.706574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.707252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.707911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.709805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e8a263-1f24-4846-964e-a14c129e4deb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.724184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 833/1666] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:09 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:09.728649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.749973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f459a-5102-46e8-b36e-fc1f0fff00d8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:09.755760 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:09.756963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.757608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.758105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] PATH_INFO: `/v3/projects/8a8a976443844a2e91f3f6fe212ac109/users/2645a999f3064c9bbda415619ba766e1/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.760610 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8a8a976443844a2e91f3f6fe212ac109, user_id=2645a999f3064c9bbda415619ba766e1, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:09.763370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:09.775951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 834/1667] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:09 2022] PUT /identity/v3/projects/8a8a976443844a2e91f3f6fe212ac109/users/2645a999f3064c9bbda415619ba766e1/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:09.786459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:09.786768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:09.787040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:09.817001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] MFA Rules not processed for user `2645a999f3064c9bbda415619ba766e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:09.818265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.914520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 834/1668] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:09 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:09.925500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.927259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:09.946701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:09.988546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:09.993456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.028149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b8415a-5203-40cf-9960-9dc9534c4131 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.032391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.033004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.033282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.033555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.045975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.048798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea68a-8f25-401b-af38-34f2d32c2ce0 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.059618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:10.060493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 835/1669] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:09 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:10.082949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d5d934-266f-49cd-bbd4-369e691f2606 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.089308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.090077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.090482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.090873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] PATH_INFO: `/v3/projects/670c32a6a94642e2ae6f21c887239fd0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.092246 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=670c32a6a94642e2ae6f21c887239fd0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.094387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.163059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '670c32a6a94642e2ae6f21c887239fd0', 'request_id': 'req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.170044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '670c32a6a94642e2ae6f21c887239fd0', 'request_id': 'req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.178037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '670c32a6a94642e2ae6f21c887239fd0', 'request_id': 'req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.197731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] The token cache is being invalidate because project 670c32a6a94642e2ae6f21c887239fd0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:10.198019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.198772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8152ce56-e177-43b8-a5a5-f5eaee4f97d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.201125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 835/1670] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:09 2022] DELETE /identity/v3/projects/670c32a6a94642e2ae6f21c887239fd0 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:10.206199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.206681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.206773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.231978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:10.233177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.307727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 836/1671] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:10.316024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.318770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.319102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.347242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:10.348770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.358894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 836/1672] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:10.363832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.376500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c623b6ce-4fd2-4466-8571-ed352946888c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.381996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.382408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.382932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.383282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.384942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:10.385119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.387599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.401821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 837/1673] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:10.407546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.421096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b8935b-74d3-4825-965c-826d953f5898 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.429964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.431016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.431762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.432241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.433633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.436009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.494976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 837/1674] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:10.495162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.496604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.498420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.513468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab163d-a638-47e0-ba70-7b9be615094c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.577439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.578099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.578473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.578628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70cd0f8-3a52-49c3-af98-8949f580c486 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.578715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] PATH_INFO: `/v3/users/bcfa077ca70841eb86137bd39eb7a281` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.579832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcfa077ca70841eb86137bd39eb7a281)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.581747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.584763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.586168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.586485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.586791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.587463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.589374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.622217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 838/1675] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/users => generated 291 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:10.623503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.625337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.636541 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:10.637400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab1727a-d0be-4a2a-9d06-ce8de03fb38d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.638321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcfa077ca70841eb86137bd39eb7a281'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.642949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.643577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.644029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.644316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] PATH_INFO: `/v3/users/e3da95d1ba214ed99eaf5f17fc3a39ee` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.645445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3da95d1ba214ed99eaf5f17fc3a39ee)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.646381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcfa077ca70841eb86137bd39eb7a281'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.647372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.654792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcfa077ca70841eb86137bd39eb7a281'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.655429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcfa077ca70841eb86137bd39eb7a281'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.656086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcfa077ca70841eb86137bd39eb7a281'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.674613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 838/1676] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:10 2022] DELETE /identity/v3/users/bcfa077ca70841eb86137bd39eb7a281 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:10.678273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.679640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.691941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:10.704495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3da95d1ba214ed99eaf5f17fc3a39ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.711318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3da95d1ba214ed99eaf5f17fc3a39ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.712556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3da95d1ba214ed99eaf5f17fc3a39ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.720563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3da95d1ba214ed99eaf5f17fc3a39ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.721350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a47f6960-cfaa-4ec4-b169-e60ad92b994d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3da95d1ba214ed99eaf5f17fc3a39ee'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:10.739810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 839/1677] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:10 2022] DELETE /identity/v3/users/e3da95d1ba214ed99eaf5f17fc3a39ee => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:10.741158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89317c-6b4b-4afd-a8fa-26e2a05b00c7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.741449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89317c-6b4b-4afd-a8fa-26e2a05b00c7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.741722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de89317c-6b4b-4afd-a8fa-26e2a05b00c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.750136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7486195d-9ef8-4382-a5f4-5337ee9322d8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.760857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-de89317c-6b4b-4afd-a8fa-26e2a05b00c7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:10.762044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-de89317c-6b4b-4afd-a8fa-26e2a05b00c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.810679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.811300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.811580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.812030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.812487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.814083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.819774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 840/1678] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:10.823240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.823240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.823240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.830923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 839/1679] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:10.831644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.833333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.845933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:10.847183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.857518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 841/1680] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:10.858166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.859605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:10.872685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393aaf8-f74e-4328-a2f2-c9adf297dec2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.877691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.878286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.878597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.878869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.879507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.881158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.892004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 842/1681] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:10.895857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.898283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc52db-a687-4a35-a256-e005af24f02e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.903324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.903639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.904045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.904389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.907192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3978c0c8-5626-4f95-98ee-400050378164 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.911004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.911618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.911893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.912165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.915027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.917411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.923661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.925097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5869b9b0-cbce-48c9-8974-8f45cba21fb7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.928603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 840/1682] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:10.929434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:10.936527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 843/1683] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:10.941122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea316053-2c98-44d6-9270-a2572aa13dd6 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:10.946263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:10.947377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:10.947743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:10.948086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] PATH_INFO: `/v3/projects/0bf64ce6cafd4a20be07d0a1792a6484/users/8fee4847edfa4b298bb962314a054f37/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:10.951595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:10.969636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bf64ce6cafd4a20be07d0a1792a6484, user_id=8fee4847edfa4b298bb962314a054f37, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:10.972665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:10.990227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 841/1684] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:10 2022] PUT /identity/v3/projects/0bf64ce6cafd4a20be07d0a1792a6484/users/8fee4847edfa4b298bb962314a054f37/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:11.001955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.002245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.003000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.054538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] MFA Rules not processed for user `8fee4847edfa4b298bb962314a054f37`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:11.055851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:11.135909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 844/1685] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:11 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:11.140423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.215214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94402243-786c-4d27-a700-54b69fbb6ea5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.219960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.220567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.220844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.221112 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.221755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.223684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.256281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 842/1686] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:11 2022] POST /identity/v3/projects => generated 383 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:11.257500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.271312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021620e9-92f3-4e7c-b91b-aa58a1dccf0d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.280335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:11.338788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.339418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.339699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.339971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.340675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.342810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.366194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 845/1687] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:11 2022] POST /identity/v3/users => generated 298 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:11.370891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.382609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fc570-baff-4412-83de-e4d222df75df admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.388346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.388994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.389302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.389601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.391114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.392734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.408148 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 843/1688] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:11.409636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.423022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcdcd1a-f093-4176-beb8-d6e4310df61e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.427974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.428631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.428913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.429194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] PATH_INFO: `/v3/projects/506aba0c4c4943af86662ce54a0242b3/users/bce903ee68e2418aaa4d721f8a5b1065/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.446355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=506aba0c4c4943af86662ce54a0242b3, user_id=bce903ee68e2418aaa4d721f8a5b1065, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.449215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.463126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 846/1689] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:11 2022] PUT /identity/v3/projects/506aba0c4c4943af86662ce54a0242b3/users/bce903ee68e2418aaa4d721f8a5b1065/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:11.467073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.478274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eaee9-8df7-4643-a655-be28f1b1113d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.537195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.537809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.538151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.538430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.539078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.540890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f362fcc-87bd-46af-8418-ece3218af451 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.557549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 844/1690] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:11 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:11.558836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.576554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab1b932-7832-4084-bd6b-a668a482d4ba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.583397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.584707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.585204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.585731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] PATH_INFO: `/v3/projects/506aba0c4c4943af86662ce54a0242b3/users/bce903ee68e2418aaa4d721f8a5b1065/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.588398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506aba0c4c4943af86662ce54a0242b3, user_id=bce903ee68e2418aaa4d721f8a5b1065, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.592821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.617260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 847/1691] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:11 2022] PUT /identity/v3/projects/506aba0c4c4943af86662ce54a0242b3/users/bce903ee68e2418aaa4d721f8a5b1065/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:11.618787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.619249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.619669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.673510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] MFA Rules not processed for user `bce903ee68e2418aaa4d721f8a5b1065`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:11.674906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:11.739422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:11.771123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.773158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:11.777392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 845/1692] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:11 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:11.791556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.792549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:11.873679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c328e741-fb24-4dbf-9764-c8bcc98d9065 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.878533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.879223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.879579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.879763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.882009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad8b16e-0fc4-4187-b790-b33d1ae0aec2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.894063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.894262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.895094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.895348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] PATH_INFO: `/v3/projects/97d496cc0636403784a6fd78111a97f8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.896751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d496cc0636403784a6fd78111a97f8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.896971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.898475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.899298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.912579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 846/1693] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:11 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:11.913732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:11.916316 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:11.932729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4b54ec-f739-4e7a-a122-9a7534bb1e54 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:11.939817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:11.940772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:11.941257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:11.941539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] PATH_INFO: `/v3/projects/12b73748d6404120b1606bd318c32bf4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:11.944600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b73748d6404120b1606bd318c32bf4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:11.947407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:11.956987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d496cc0636403784a6fd78111a97f8', 'request_id': 'req-2a71e97e-e423-444b-af1e-04679c539263'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:11.969181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d496cc0636403784a6fd78111a97f8', 'request_id': 'req-2a71e97e-e423-444b-af1e-04679c539263'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:11.992612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d496cc0636403784a6fd78111a97f8', 'request_id': 'req-2a71e97e-e423-444b-af1e-04679c539263'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.002656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=88,keystone:INSERT=6,keystone:DELETE=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:12.018525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] The token cache is being invalidate because project 97d496cc0636403784a6fd78111a97f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:12.018818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.019458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.019756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b73748d6404120b1606bd318c32bf4', 'request_id': 'req-e306f046-9337-4daf-bb31-3dee20232a8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.021224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 848/1694] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:11 2022] DELETE /identity/v3/projects/97d496cc0636403784a6fd78111a97f8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:12.024909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:12.026252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.027066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b73748d6404120b1606bd318c32bf4', 'request_id': 'req-e306f046-9337-4daf-bb31-3dee20232a8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.034342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b73748d6404120b1606bd318c32bf4', 'request_id': 'req-e306f046-9337-4daf-bb31-3dee20232a8b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.053311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] The token cache is being invalidate because project 12b73748d6404120b1606bd318c32bf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:12.053693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.054884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.057715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 847/1695] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:11 2022] DELETE /identity/v3/projects/12b73748d6404120b1606bd318c32bf4 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:12.061546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:12.063006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.108187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a71e97e-e423-444b-af1e-04679c539263 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:12.132094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306f046-9337-4daf-bb31-3dee20232a8b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:12.137068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:12.137690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.137970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.138250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] PATH_INFO: `/v3/users/0fdc06360aa14f60bf1a63c3a8a39619` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.139400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdc06360aa14f60bf1a63c3a8a39619)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:12.141324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:12.146573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:12.167458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:12.168099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.168402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.168699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] PATH_INFO: `/v3/users/2645a999f3064c9bbda415619ba766e1` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.169855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2645a999f3064c9bbda415619ba766e1)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:12.171898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:12.197810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdc06360aa14f60bf1a63c3a8a39619'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.205913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdc06360aa14f60bf1a63c3a8a39619'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.215842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdc06360aa14f60bf1a63c3a8a39619'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.216483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdc06360aa14f60bf1a63c3a8a39619'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.217153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdc06360aa14f60bf1a63c3a8a39619'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.225882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2645a999f3064c9bbda415619ba766e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.246284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2645a999f3064c9bbda415619ba766e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.247783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2645a999f3064c9bbda415619ba766e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.248697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 848/1696] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:12 2022] DELETE /identity/v3/users/0fdc06360aa14f60bf1a63c3a8a39619 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:12.251862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:12.260170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2645a999f3064c9bbda415619ba766e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.260631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.261901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2645a999f3064c9bbda415619ba766e1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.302729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 849/1697] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:12 2022] DELETE /identity/v3/users/2645a999f3064c9bbda415619ba766e1 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:12.361552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24590861-e4b7-4ec1-b740-942c8ee1d48d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:12.467100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:12.467842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.468793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.469081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.471321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.590329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:12.592945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:12.594588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:12.607047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 849/1698] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:12 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:12.664321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:12.667415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.713169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:12.735621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:12.740170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:12.776211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e51825-90f9-4827-9cbb-96cc5e28a169 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:12.782362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:12.783198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.783684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.784046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] PATH_INFO: `/v3/projects/8a8a976443844a2e91f3f6fe212ac109` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.785672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] RBAC: Authorizing `identity:delete_project(project_id=8a8a976443844a2e91f3f6fe212ac109)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:12.788373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:12.856019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99aed7f-3173-4bc1-a9ba-ed9ea665a98b service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:12.884453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8a8a976443844a2e91f3f6fe212ac109', 'request_id': 'req-1823d005-fed5-4453-a48b-01e9f26c16da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.892595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8a8a976443844a2e91f3f6fe212ac109', 'request_id': 'req-1823d005-fed5-4453-a48b-01e9f26c16da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.899094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8a8a976443844a2e91f3f6fe212ac109', 'request_id': 'req-1823d005-fed5-4453-a48b-01e9f26c16da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.917593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] The token cache is being invalidate because project 8a8a976443844a2e91f3f6fe212ac109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:12.917888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.926322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1823d005-fed5-4453-a48b-01e9f26c16da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:12.928701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 850/1699] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:12 2022] DELETE /identity/v3/projects/8a8a976443844a2e91f3f6fe212ac109 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:12.934549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa6ad-f08e-4a70-b7dc-dad5a866722d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.934839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa6ad-f08e-4a70-b7dc-dad5a866722d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.935389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9aa6ad-f08e-4a70-b7dc-dad5a866722d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.958615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:12.959270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:12.959628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:12.959898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] PATH_INFO: `/v3/projects/6f97908764f441628d46a24049cbe7ad` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:12.960996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f97908764f441628d46a24049cbe7ad)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:12.963450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:12.984034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ca9aa6ad-f08e-4a70-b7dc-dad5a866722d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:12.985510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca9aa6ad-f08e-4a70-b7dc-dad5a866722d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.002254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 851/1700] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:13.003805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.004031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.004267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.018691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f97908764f441628d46a24049cbe7ad', 'request_id': 'req-cb8f3048-37e2-4b68-bc75-cbda7c03998e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.029677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f97908764f441628d46a24049cbe7ad', 'request_id': 'req-cb8f3048-37e2-4b68-bc75-cbda7c03998e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.029995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:13.031700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.049969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f97908764f441628d46a24049cbe7ad', 'request_id': 'req-cb8f3048-37e2-4b68-bc75-cbda7c03998e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.070124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] The token cache is being invalidate because project 6f97908764f441628d46a24049cbe7ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:13.070434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.071261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.075026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 850/1701] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:12 2022] DELETE /identity/v3/projects/6f97908764f441628d46a24049cbe7ad => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:13.076749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.082553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.106351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 852/1702] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:13.110578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.112049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.125125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4001dbe8-2462-46ac-b9bd-43a95f94d637 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.128464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:13.129854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.130517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.130801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.131077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] PATH_INFO: `/v3/users/b5dfc4f5392c482e8c284bd55da80d93` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.132423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5dfc4f5392c482e8c284bd55da80d93)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.136048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.167074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8f3048-37e2-4b68-bc75-cbda7c03998e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.174141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.174616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.174957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.175302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] PATH_INFO: `/v3/users/8fee4847edfa4b298bb962314a054f37` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.176971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fee4847edfa4b298bb962314a054f37)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.179238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.227053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:13.246938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fee4847edfa4b298bb962314a054f37'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.255953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fee4847edfa4b298bb962314a054f37'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.257209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5dfc4f5392c482e8c284bd55da80d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.269907 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fee4847edfa4b298bb962314a054f37'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.271024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fee4847edfa4b298bb962314a054f37'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.272353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fee4847edfa4b298bb962314a054f37'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.287972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5dfc4f5392c482e8c284bd55da80d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.289122 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5dfc4f5392c482e8c284bd55da80d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.305263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 851/1703] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:13 2022] DELETE /identity/v3/users/8fee4847edfa4b298bb962314a054f37 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:13.307063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.308521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.310429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5dfc4f5392c482e8c284bd55da80d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.311108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e2fbbbe0-ff40-4909-acdc-9c6629782cf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5dfc4f5392c482e8c284bd55da80d93'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.351640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 853/1704] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:13 2022] DELETE /identity/v3/users/b5dfc4f5392c482e8c284bd55da80d93 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:13.357168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.357470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.358314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.385245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:13.387031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.391781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034fe5d6-e1cc-44a5-ae6e-0ee2feaf4eb8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.496608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.497229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.497511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.497786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.498454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:13.498958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.500666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.504621 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:13.504762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 854/1705] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:13.522004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.527394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 852/1706] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:13.528134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.530674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.533085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.551020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e111d6c-24d8-4873-807a-87eda9142464 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.558648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.559326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.559669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.559951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.560844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.563472 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.602427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b568ed-338c-4476-b317-e05603c1fd8c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.606660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.607273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.607551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.607864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.609616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 853/1707] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:13 2022] POST /identity/v3/projects => generated 373 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:13.615975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.618616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.620349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.632328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 855/1708] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:13.717354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c69ef9-a413-4504-9c79-48628fb9edda admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.722238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.722885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.723171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.723451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.724095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.726051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.749118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 854/1709] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:13 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:13.753221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.763213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2256f018-6367-4b71-8042-a0a9b01033ac service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.768024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.768628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.768910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.769186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.769814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.771453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.782293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 856/1710] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:13.786370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.797379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4862d932-6a2c-4f07-bb24-a2373849a91e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.802376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.803016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.803298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.803572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.804215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.806157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.819199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 855/1711] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:13.824466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.834839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee4623-9dc5-457c-bf5c-f10f5b018aaf admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.840602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.841257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.841542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.841815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] PATH_INFO: `/v3/projects/cb9db0499f004e3b8e4d7196111611fb/users/d4cd13afb01a4192a3c7cf3c0a69943b/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.848931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:13.850523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.861034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb9db0499f004e3b8e4d7196111611fb, user_id=d4cd13afb01a4192a3c7cf3c0a69943b, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.865313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c703f7-0f2e-4495-b287-000c9c53582b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.867269 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae863cf-6ccd-4e41-8055-1979c41961f1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:13.873497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:13.874140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.874425 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.874727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] PATH_INFO: `/v3/projects/e1ece61fbf7541c8bae1f73443a799cb` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.875875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1ece61fbf7541c8bae1f73443a799cb)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:13.877772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:13.888543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 857/1712] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:13 2022] PUT /identity/v3/projects/cb9db0499f004e3b8e4d7196111611fb/users/d4cd13afb01a4192a3c7cf3c0a69943b/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:13.903113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:13.903400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:13.903675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:13.954144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] MFA Rules not processed for user `d4cd13afb01a4192a3c7cf3c0a69943b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:13.956910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:13.968409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1ece61fbf7541c8bae1f73443a799cb', 'request_id': 'req-db4dfa6c-d329-4d44-be98-01202b65faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.973916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:13.983241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1ece61fbf7541c8bae1f73443a799cb', 'request_id': 'req-db4dfa6c-d329-4d44-be98-01202b65faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:13.990287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1ece61fbf7541c8bae1f73443a799cb', 'request_id': 'req-db4dfa6c-d329-4d44-be98-01202b65faef'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.021829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] The token cache is being invalidate because project e1ece61fbf7541c8bae1f73443a799cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:14.022247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.023032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-db4dfa6c-d329-4d44-be98-01202b65faef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.025978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 856/1713] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:13 2022] DELETE /identity/v3/projects/e1ece61fbf7541c8bae1f73443a799cb => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:14.031754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.032133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.032457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.052368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 858/1714] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:14.053496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.057943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:14.059291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.059388 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.097170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:14.142273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 857/1715] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:14.143361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.145086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.171269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad62127d-729d-4d8d-94d6-7fe8872c50b8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.176852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.177775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.178066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.178346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] PATH_INFO: `/v3/projects/0bf64ce6cafd4a20be07d0a1792a6484` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.179628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bf64ce6cafd4a20be07d0a1792a6484)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.185006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.208365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2e819b-c89a-4a4d-8cc5-aaed4667d42a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.213301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.214574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.214823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.215095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.226475 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.228700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f12812-301c-4652-bf75-52d65926cbed service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.239429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 858/1716] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:14.240802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.241130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.241593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.253582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bf64ce6cafd4a20be07d0a1792a6484', 'request_id': 'req-a353c233-2477-4eb3-b131-99848a3cdf02'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.262124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bf64ce6cafd4a20be07d0a1792a6484', 'request_id': 'req-a353c233-2477-4eb3-b131-99848a3cdf02'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.263131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:14.264470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.268872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bf64ce6cafd4a20be07d0a1792a6484', 'request_id': 'req-a353c233-2477-4eb3-b131-99848a3cdf02'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.302949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] The token cache is being invalidate because project 0bf64ce6cafd4a20be07d0a1792a6484 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:14.303536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.303794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.309734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 859/1717] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:14 2022] DELETE /identity/v3/projects/0bf64ce6cafd4a20be07d0a1792a6484 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:14.310915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.312448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.344011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 859/1718] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:14.348973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.372542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:14.373459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6937a7-32e2-4e90-82a1-a6345bbd189f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.382993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.384403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.384970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.384970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.385852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:14.386066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.388192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.401734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a353c233-2477-4eb3-b131-99848a3cdf02 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.419181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.419181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.419181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.419181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] PATH_INFO: `/v3/users/bce903ee68e2418aaa4d721f8a5b1065` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.419688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] RBAC: Authorizing `identity:delete_user(user_id=bce903ee68e2418aaa4d721f8a5b1065)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.419796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 860/1719] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:14.419928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.426280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.439113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1bc834-ad19-4cee-929d-80aa7d4028c2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.444861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.445492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.446935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.446935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.447665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.449953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.469575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bce903ee68e2418aaa4d721f8a5b1065'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.478561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bce903ee68e2418aaa4d721f8a5b1065'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.479306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bce903ee68e2418aaa4d721f8a5b1065'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.487953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 861/1720] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:14 2022] POST /identity/v3/projects => generated 373 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:14.493404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.493928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bce903ee68e2418aaa4d721f8a5b1065'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.494923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bce903ee68e2418aaa4d721f8a5b1065'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:14.497137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:14.548959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 860/1721] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:14 2022] DELETE /identity/v3/users/bce903ee68e2418aaa4d721f8a5b1065 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:14.603559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:14.633158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d8f97f-107f-4f28-841c-5deaa41b0b5c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.740963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.742374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.742685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.743518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.744369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.746367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.778877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 862/1722] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:14 2022] POST /identity/v3/users => generated 287 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:14.780187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.794853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a6197-ba1a-497c-b53a-cc91250636eb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.801036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.801728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.802018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.802301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.802956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.804762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.819225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 861/1723] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:14.823216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.835361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799d9c-080e-4a2c-8cd4-630b395791e4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.840409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.841023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.841303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.841578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.842213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.843912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7b73bc-4c10-4f11-95f6-b7797deb56d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.855105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 863/1724] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:14.859255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:14.870894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d27d0dd-da80-472c-a34d-7f6388b3cdba admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:14.876405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:14.876950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.877222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.877505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] PATH_INFO: `/v3/projects/1d3333e0571942109b2808bf69b91433/users/4c60eb8b9669485d9a97d2adccb781ea/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.894929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d3333e0571942109b2808bf69b91433, user_id=4c60eb8b9669485d9a97d2adccb781ea, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:14.897100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:14.912133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 862/1725] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:14 2022] PUT /identity/v3/projects/1d3333e0571942109b2808bf69b91433/users/4c60eb8b9669485d9a97d2adccb781ea/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:14.925152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:14.925448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:14.925729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:14.956273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] MFA Rules not processed for user `4c60eb8b9669485d9a97d2adccb781ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:14.957570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.041511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 864/1726] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:14 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:15.045765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.109307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e81aafc-7ff8-49d4-8b18-7845c7ceb619 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.114315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.114950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.115242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.115578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.116148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.118212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.123220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.124971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.145750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfcf78-aacc-48b8-9372-c4c0cb700602 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.151262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.151888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.152173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.152469 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] PATH_INFO: `/v3/projects/506aba0c4c4943af86662ce54a0242b3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.153594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] RBAC: Authorizing `identity:delete_project(project_id=506aba0c4c4943af86662ce54a0242b3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.154474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 863/1727] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:15.155229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.155652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.170432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:15.222521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506aba0c4c4943af86662ce54a0242b3', 'request_id': 'req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:15.225734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:15.232120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506aba0c4c4943af86662ce54a0242b3', 'request_id': 'req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:15.232657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b800b1b2-f7d4-47b6-b9ad-abb1d228244f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.239611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506aba0c4c4943af86662ce54a0242b3', 'request_id': 'req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:15.268173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] The token cache is being invalidate because project 506aba0c4c4943af86662ce54a0242b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:15.268730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:15.269151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1b09fc0-3aab-4cbe-ae0a-2581bb7d082b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:15.271111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 865/1728] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:15 2022] DELETE /identity/v3/projects/506aba0c4c4943af86662ce54a0242b3 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:15.276583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71c2-1af6-4af8-af16-373ca9457801 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.276950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71c2-1af6-4af8-af16-373ca9457801 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.277213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7e71c2-1af6-4af8-af16-373ca9457801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.301477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-af7e71c2-1af6-4af8-af16-373ca9457801 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:15.302769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-af7e71c2-1af6-4af8-af16-373ca9457801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.311558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.312180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.312461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.312737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.313375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.315382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.343416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 864/1729] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:15.347081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.350728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.365827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1afe0bd-3d7a-44b6-b9c8-9256c9ec2d95 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.372810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.373428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.376011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.376011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.376591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.378569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.380309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 866/1730] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:15.387811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.388157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.388468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.393027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 865/1731] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:15.394373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.408070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a1c5c0-4f8c-4380-8adb-e18d335d3c85 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.410064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:15.411380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.412802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.413591 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.413884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.414327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] PATH_INFO: `/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82/users/53afaa3d77714a92b333fcc3fcf9da53/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.424147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 867/1732] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:15.428543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.437802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed265ab-3911-45a4-854b-747a92bbc1e1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.446892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.447180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.447387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.447665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.448369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:15.448670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.450330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.454729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f330f7e4a5fb40ba960544c4f1a75a82, user_id=53afaa3d77714a92b333fcc3fcf9da53, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.457395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.473725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 868/1733] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:12:15.476930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 866/1734] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:15 2022] PUT /identity/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82/users/53afaa3d77714a92b333fcc3fcf9da53/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:15.479605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.491327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.502074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fb1b2-dbef-4d3c-b508-1238519c3071 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.569519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29ee72-0da9-43c6-8191-4d66f6c6e561 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.574908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.575545 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.575809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.576173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.576831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.578492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.580298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.580943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.581225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.581502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.582159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.584092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.589105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 867/1735] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:15.593240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.603828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba31234d-3df0-4ec6-b042-9014d8504e9e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.608531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.609168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.609448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.609721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] PATH_INFO: `/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82/users/53afaa3d77714a92b333fcc3fcf9da53/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.611631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f330f7e4a5fb40ba960544c4f1a75a82, user_id=53afaa3d77714a92b333fcc3fcf9da53, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.616116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 869/1736] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/projects => generated 383 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:15.616258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ec3060-11cb-422f-975c-68de6a74073b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.617242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.621400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:15.627440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 868/1737] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:15 2022] PUT /identity/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82/users/53afaa3d77714a92b333fcc3fcf9da53/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:15.639747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.640036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.640312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.669705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] MFA Rules not processed for user `53afaa3d77714a92b333fcc3fcf9da53`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:15.670152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:15.671013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.690377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b78445-09a1-4513-b268-6e8a93b9a28d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.753891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 869/1738] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:15.754084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.754351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.754735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.754952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.755665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.758331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.763884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.765303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.790266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 870/1739] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/users => generated 292 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:15.794638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.804481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f04233f-5c7b-4476-89b6-7b67ea609b18 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.809659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.810251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.810549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.810820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.811442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.813052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.823387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 871/1740] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:15.824290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257f1351-bedf-48a1-bada-1677e4c8f5fe None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.827198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.828143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.828751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.829025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.829294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.831029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.837853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a323bf7-d41c-4020-8acd-e53fe3dbec24 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.842465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.843047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.843346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.843620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.844237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.845803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.856367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 872/1741] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:15.860551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:15.870935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a05a8d-9851-41d0-8bb0-17009c336dc7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:15.875924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:15.876552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.876831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.877113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] PATH_INFO: `/v3/projects/7351e04930c44ac6b8b7e6e743fc85d9/users/9b7b073809104331a2f535d8104d6d65/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.890709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.892085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7351e04930c44ac6b8b7e6e743fc85d9, user_id=9b7b073809104331a2f535d8104d6d65, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:15.892229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36c35b7-0987-4c3a-9baf-bfe206d98e7f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.894719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:15.901985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 870/1742] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:12:15.907936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 873/1743] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:15 2022] PUT /identity/v3/projects/7351e04930c44ac6b8b7e6e743fc85d9/users/9b7b073809104331a2f535d8104d6d65/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:15.919430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:15.919720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:15.919994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:15.948811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] MFA Rules not processed for user `9b7b073809104331a2f535d8104d6d65`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:15.950022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:15.975855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:16.016054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 871/1744] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:16.021306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.108023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3be92-b562-4d9a-9f30-4044c7621753 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.113754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.114480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.114774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.115189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.115899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.118895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.153748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 874/1745] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:16 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:16.155104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.166668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81bb2021-4720-4415-9571-4869d5708e98 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.238703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.239316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.239596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.239871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.240508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.242606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.271996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 872/1746] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:16 2022] POST /identity/v3/users => generated 298 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:16.277230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.295065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc28d46-5cea-4eb2-83ed-90e5604b6ee9 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.302125 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.303251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.304223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.304223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.304903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.307021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.318874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 875/1747] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:16.324093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.338211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114bab-1544-4b05-8207-5c9a8d0393bd admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.346823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.347522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.347799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.348231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] PATH_INFO: `/v3/projects/8b41804396e9486d8eb45ab686120dfa/users/008c7743191c40d78cf05198dc74c1b5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.364488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b41804396e9486d8eb45ab686120dfa, user_id=008c7743191c40d78cf05198dc74c1b5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.367176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.379799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 873/1748] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:16 2022] PUT /identity/v3/projects/8b41804396e9486d8eb45ab686120dfa/users/008c7743191c40d78cf05198dc74c1b5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:16.384370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.394240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28ba3cc-63c1-4d40-a061-04977c953ec3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.402687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:16.455618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.456240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.456533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.456806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.457471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.459116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132b7620-d598-4ae6-8427-21dfcad2fce4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.470794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 876/1749] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:16 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:16.476535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.488524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bd25f8-48eb-44cb-9549-07c4dc19f081 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.496126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.498291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.498291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.498291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] PATH_INFO: `/v3/projects/8b41804396e9486d8eb45ab686120dfa/users/008c7743191c40d78cf05198dc74c1b5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.514673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b41804396e9486d8eb45ab686120dfa, user_id=008c7743191c40d78cf05198dc74c1b5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.522816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.556124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 874/1750] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:16 2022] PUT /identity/v3/projects/8b41804396e9486d8eb45ab686120dfa/users/008c7743191c40d78cf05198dc74c1b5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:16.557471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.557919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.558344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.608394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] MFA Rules not processed for user `008c7743191c40d78cf05198dc74c1b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:16.609819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:16.700509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 877/1751] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:16.708891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:16.710998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:16.776179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5127f27c-d280-409e-9bb0-d4baf5b3334d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:16.780217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:16.780847 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:16.781129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:16.781506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:16.841086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:16.870915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:16.872360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a84bb0-9443-409c-baeb-9d1630a5c432 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:16.881729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 875/1752] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:19.146911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:19.148449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:19.243544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfd8bc5-ba17-47c2-8ddd-c3a64bf19a0b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:19.250569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:19.251791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:19.252222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:19.252657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] PATH_INFO: `/v3/users/d4cd13afb01a4192a3c7cf3c0a69943b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:19.254280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4cd13afb01a4192a3c7cf3c0a69943b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:19.261643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:19.325423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:19.341018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4cd13afb01a4192a3c7cf3c0a69943b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.353519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4cd13afb01a4192a3c7cf3c0a69943b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.363838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4cd13afb01a4192a3c7cf3c0a69943b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.364669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4cd13afb01a4192a3c7cf3c0a69943b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.365429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4cd13afb01a4192a3c7cf3c0a69943b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.386921 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 878/1753] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:19 2022] DELETE /identity/v3/users/d4cd13afb01a4192a3c7cf3c0a69943b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:19.392234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:19.392745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:19.392905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:19.413797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:19.415419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:19.481541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 876/1754] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:19.488759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:19.490229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:19.558490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2fa2fd-feb8-4b04-b27c-bd82f9a4dda0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:19.562283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:19.562918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:19.563197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:19.563468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:19.573176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:19.574647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:19.584239 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 879/1755] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:19.866589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:19.868309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:19.902002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7808642f-bdd1-4bd3-a25d-3a20277f366b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:19.904790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:19.906274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:19.907117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:19.908024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:19.908256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:19.908533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] PATH_INFO: `/v3/projects/cb9db0499f004e3b8e4d7196111611fb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:19.910398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb9db0499f004e3b8e4d7196111611fb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:19.912395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:19.921298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc891a-b4ac-4805-9ad6-402b05b3fc44 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:19.927950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:19.928710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:19.929090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:19.929466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] PATH_INFO: `/v3/users/4c60eb8b9669485d9a97d2adccb781ea` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:19.930834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4c60eb8b9669485d9a97d2adccb781ea)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:19.932869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:19.979214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb9db0499f004e3b8e4d7196111611fb', 'request_id': 'req-40c2e320-2b09-456d-a1a2-5462089e6298'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.990281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb9db0499f004e3b8e4d7196111611fb', 'request_id': 'req-40c2e320-2b09-456d-a1a2-5462089e6298'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:19.997694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb9db0499f004e3b8e4d7196111611fb', 'request_id': 'req-40c2e320-2b09-456d-a1a2-5462089e6298'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.006774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4c60eb8b9669485d9a97d2adccb781ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.017363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4c60eb8b9669485d9a97d2adccb781ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.026931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4c60eb8b9669485d9a97d2adccb781ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.027635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4c60eb8b9669485d9a97d2adccb781ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.028407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4c60eb8b9669485d9a97d2adccb781ea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.034807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] The token cache is being invalidate because project cb9db0499f004e3b8e4d7196111611fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:20.035108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.035808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-40c2e320-2b09-456d-a1a2-5462089e6298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.037581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 877/1756] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:19 2022] DELETE /identity/v3/projects/cb9db0499f004e3b8e4d7196111611fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:20.049664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 880/1757] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:19 2022] DELETE /identity/v3/users/4c60eb8b9669485d9a97d2adccb781ea => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:20.054711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:20.055830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:20.056374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:20.083446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:20.085374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:20.155866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:20.170667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 878/1758] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:20.186878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:20.189325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:20.246416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:20.283888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5439a18c-7c86-467b-b303-70f880b449f0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:20.287502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:20.288109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:20.288383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:20.288652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:20.299006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:20.300442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:20.310093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 881/1759] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:20.604789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:20.606271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:20.618759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d409cd4-8d75-45c0-929a-9867276e7810 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:20.623539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:20.624176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:20.624457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:20.624736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] PATH_INFO: `/v3/projects/1d3333e0571942109b2808bf69b91433` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:20.625851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d3333e0571942109b2808bf69b91433)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:20.627790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:20.674460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d3333e0571942109b2808bf69b91433', 'request_id': 'req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.681288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d3333e0571942109b2808bf69b91433', 'request_id': 'req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.687025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d3333e0571942109b2808bf69b91433', 'request_id': 'req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.701478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] The token cache is being invalidate because project 1d3333e0571942109b2808bf69b91433 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:20.701793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.702489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.704062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 879/1760] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:20 2022] DELETE /identity/v3/projects/1d3333e0571942109b2808bf69b91433 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:20.708134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:20.717974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5e0bf-ec1d-4fef-83bb-e12b26b867e4 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:20.774942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:20.775532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:20.775833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:20.776109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] PATH_INFO: `/v3/users/53afaa3d77714a92b333fcc3fcf9da53` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:20.777223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] RBAC: Authorizing `identity:delete_user(user_id=53afaa3d77714a92b333fcc3fcf9da53)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:20.779299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:20.826320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53afaa3d77714a92b333fcc3fcf9da53'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.833214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53afaa3d77714a92b333fcc3fcf9da53'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.840786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53afaa3d77714a92b333fcc3fcf9da53'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.841417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53afaa3d77714a92b333fcc3fcf9da53'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.842030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ccc88bb9-8e23-4178-9691-d80e153a716b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53afaa3d77714a92b333fcc3fcf9da53'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:20.861321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 882/1761] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:20 2022] DELETE /identity/v3/users/53afaa3d77714a92b333fcc3fcf9da53 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:21.130704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.132234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:21.145999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f11c5d8-25e8-44f2-b3ed-f66ba30a1735 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.211162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.211796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.212082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.212361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] PATH_INFO: `/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.213510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f330f7e4a5fb40ba960544c4f1a75a82)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.215451 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.261119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f330f7e4a5fb40ba960544c4f1a75a82', 'request_id': 'req-c6ed828c-8bc9-461c-9495-f8a12d9cea48'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:21.267328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f330f7e4a5fb40ba960544c4f1a75a82', 'request_id': 'req-c6ed828c-8bc9-461c-9495-f8a12d9cea48'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:21.272729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f330f7e4a5fb40ba960544c4f1a75a82', 'request_id': 'req-c6ed828c-8bc9-461c-9495-f8a12d9cea48'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:21.286375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] The token cache is being invalidate because project f330f7e4a5fb40ba960544c4f1a75a82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:21.286625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:21.287235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c6ed828c-8bc9-461c-9495-f8a12d9cea48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:21.288853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 880/1762] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:21 2022] DELETE /identity/v3/projects/f330f7e4a5fb40ba960544c4f1a75a82 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:21.293701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.294158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.294294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.316076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:21.317174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:21.373396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:21.392582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 883/1763] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:21.400145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.400427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.400699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.420766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:21.422019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:21.432782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 881/1764] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:21.437500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.448515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a3948a-e9c8-4b61-8bfc-5b61177431ec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.453970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.454726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.455083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.455428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.456140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:21.456454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.458083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.469601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 884/1765] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:21.473833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.479609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:21.483434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a257bc05-1df1-4ff0-8313-408dbca53aeb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.489338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.490107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.490599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.490950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.491758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.494356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.521137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 882/1766] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:21 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:21.525656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.535444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427807ae-ac28-4d97-a984-0ff3ed798d73 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.591181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.591593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.591871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.592202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.592793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.594664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.620572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 885/1767] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:21.621729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.632265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980803d5-c897-4951-aec8-7b8f118b5007 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.637840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.638503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.638787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.639056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.639688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.641288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.652174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 883/1768] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:21.656266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.667382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3aaca0-5ac3-46d6-84ec-d634ac8bd43f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.672075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.672941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.673218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.673484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.674118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.675742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969637a-cda8-461d-95c1-7e540a74fdf3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.689465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 886/1769] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:21.693288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.704244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2df7637-5d44-4803-b2b3-8f1120057024 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.709199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.709858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.710137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.710459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] PATH_INFO: `/v3/projects/e84f012d59214a6bb6e7f5490bfb2bfc/users/04ee886aeea740428a58a513600fc9b8/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.726279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=e84f012d59214a6bb6e7f5490bfb2bfc, user_id=04ee886aeea740428a58a513600fc9b8, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.729718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.744421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 884/1770] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:21 2022] PUT /identity/v3/projects/e84f012d59214a6bb6e7f5490bfb2bfc/users/04ee886aeea740428a58a513600fc9b8/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:21.749651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ff4-4068-4839-9e0d-f584b1707273 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.749947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ff4-4068-4839-9e0d-f584b1707273 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.750223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649d8ff4-4068-4839-9e0d-f584b1707273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.769154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-649d8ff4-4068-4839-9e0d-f584b1707273 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:21.770424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-649d8ff4-4068-4839-9e0d-f584b1707273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:21.829052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 887/1771] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:21.839045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:21.840805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:21.897134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6941e6d3-fabc-4f5c-bfa4-e7e2df1d1d3a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:21.901031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:21.901627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:21.901908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:21.902175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:21.910878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:21.912287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:21.922142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 885/1772] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:22.540549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2027-cb57-4aeb-927c-e72aa6078eae None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:22.541102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2027-cb57-4aeb-927c-e72aa6078eae None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:22.541102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40f2027-cb57-4aeb-927c-e72aa6078eae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:22.573319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d40f2027-cb57-4aeb-927c-e72aa6078eae None None] MFA Rules not processed for user `04ee886aeea740428a58a513600fc9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:22.574554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d40f2027-cb57-4aeb-927c-e72aa6078eae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:22.624930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:22.657201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 888/1773] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:22.668350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:22.669801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:22.738993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e11f3-a48d-485d-ab3a-f422558252e7 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:22.744046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:22.744733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:22.745070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:22.745363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:22.756945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:22.759229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:22.768644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 886/1774] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:27.318344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:27.318950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:27.318950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:27.339186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] MFA Rules not processed for user `04ee886aeea740428a58a513600fc9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:27.340453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:27.347573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 889/1775] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:28.968405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:28.980358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336233d-5acc-40b1-9805-175e9e423cad service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:28.984374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:28.984980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:28.985266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:28.985547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:28.987333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:29.070702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:29.095323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:29.096857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843925a-3148-4cb8-a690-2d12c48a3a82 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:29.110678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 887/1776] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:28 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:37.332747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=10 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:39.105212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:39.604118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:39.605696 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:39.619949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4077e003-5ce1-4c57-933a-c723247e7cec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:39.626513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:39.627026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:39.627310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:39.627858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] PATH_INFO: `/v3/users/9b7b073809104331a2f535d8104d6d65` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:39.629006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b7b073809104331a2f535d8104d6d65)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:39.631384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:39.687812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b7b073809104331a2f535d8104d6d65'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:39.697110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b7b073809104331a2f535d8104d6d65'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:39.707573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b7b073809104331a2f535d8104d6d65'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:39.709208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b7b073809104331a2f535d8104d6d65'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:39.710393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b7b073809104331a2f535d8104d6d65'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:39.729437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 890/1777] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:39 2022] DELETE /identity/v3/users/9b7b073809104331a2f535d8104d6d65 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:39.734003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622940af-a99f-4226-abb1-4cb898974566 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:39.734294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622940af-a99f-4226-abb1-4cb898974566 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:39.734615 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622940af-a99f-4226-abb1-4cb898974566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:39.754670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-622940af-a99f-4226-abb1-4cb898974566 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:39.755860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-622940af-a99f-4226-abb1-4cb898974566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:39.823928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 888/1778] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:39.830895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:39.832855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:39.893903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507bc169-7dec-4d80-9e33-b7ebaaf16185 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:39.898122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:39.898774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:39.899057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:39.899333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:39.908522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:39.909979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:39.920409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 891/1779] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:40.200818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622940af-a99f-4226-abb1-4cb898974566 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:40.202297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-622940af-a99f-4226-abb1-4cb898974566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:40.216062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622940af-a99f-4226-abb1-4cb898974566 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:40.220747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:40.221613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:40.221898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:40.222171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] PATH_INFO: `/v3/projects/7351e04930c44ac6b8b7e6e743fc85d9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:40.223536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7351e04930c44ac6b8b7e6e743fc85d9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:40.225439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:40.277165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7351e04930c44ac6b8b7e6e743fc85d9', 'request_id': 'req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.283515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7351e04930c44ac6b8b7e6e743fc85d9', 'request_id': 'req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.289634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7351e04930c44ac6b8b7e6e743fc85d9', 'request_id': 'req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.303406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] The token cache is being invalidate because project 7351e04930c44ac6b8b7e6e743fc85d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:40.303698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.304480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.305980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 889/1780] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:40 2022] DELETE /identity/v3/projects/7351e04930c44ac6b8b7e6e743fc85d9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:40.314569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:40.324692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1006f9d2-de02-413d-9dd9-31da3b1b82a5 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:40.384789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:40.385429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:40.385713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:40.385992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] PATH_INFO: `/v3/users/008c7743191c40d78cf05198dc74c1b5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:40.387139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] RBAC: Authorizing `identity:delete_user(user_id=008c7743191c40d78cf05198dc74c1b5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:40.389063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:40.407119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:40.450063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '008c7743191c40d78cf05198dc74c1b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.457635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '008c7743191c40d78cf05198dc74c1b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.466282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '008c7743191c40d78cf05198dc74c1b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.467039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '008c7743191c40d78cf05198dc74c1b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.467795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a2184515-a503-48c6-b9a5-b6f2fb5a0548 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '008c7743191c40d78cf05198dc74c1b5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.486540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 892/1781] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:40 2022] DELETE /identity/v3/users/008c7743191c40d78cf05198dc74c1b5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:40.791735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:40.793339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:40.809419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c208337-7ecd-48df-9da0-cfaeff9cd7d1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:40.887903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:40.888543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:40.888828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:40.889103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] PATH_INFO: `/v3/projects/8b41804396e9486d8eb45ab686120dfa` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:40.890220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b41804396e9486d8eb45ab686120dfa)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:40.892151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:40.924424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:40.943258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b41804396e9486d8eb45ab686120dfa', 'request_id': 'req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.950385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b41804396e9486d8eb45ab686120dfa', 'request_id': 'req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.955833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b41804396e9486d8eb45ab686120dfa', 'request_id': 'req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.968013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] The token cache is being invalidate because project 8b41804396e9486d8eb45ab686120dfa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:40.968242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.968887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e44305b9-a50a-4ad0-9cf9-301a5aeecec2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:40.970403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 890/1782] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:40 2022] DELETE /identity/v3/projects/8b41804396e9486d8eb45ab686120dfa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:40.976515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:40.976984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:40.977093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:40.997052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:40.998240 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.057375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 893/1783] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:41.067028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.067418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.067793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.089539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:41.090854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.098958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 891/1784] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:41.103284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.112967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23c70d-62b4-4cdb-b6d8-57b753103ae2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.117642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.118273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.118611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.118884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.119527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:41.119833 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.121583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.134201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 894/1785] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:41.138905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.149149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbd0bce-18bd-4365-b110-71c5b6db6bec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.153785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.154426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.154729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.155002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.155633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.157521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.190848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 892/1786] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:41 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:41.192136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.203078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eade73-552f-4345-b2c2-aabb80c2fd6e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.261852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.262468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.262801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.263073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.263714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.265659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.287742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 895/1787] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:41 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:41.292247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.302932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c92d1c-67cf-46ef-9683-74204c0177f6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.307699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.308296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.308579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.308854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.309493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.311130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.322311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 893/1788] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:41.326275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.337479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73d9db5-a44b-4b7b-9371-6dfdda822f59 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.342086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.342712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.342992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.343262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.343893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.345482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0693b0df-f134-4ef9-8e40-5e93949b36f3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.350832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:41.358141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 896/1789] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:41.362219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.372548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b4ab92-aba5-400d-9df8-4c06d9d7c083 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.377348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.378005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.378285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.378597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] PATH_INFO: `/v3/projects/9eb20171c1e24eb9af9a4f404f5d5a23/users/36236ddbf6c644f7a7e82fe8dffb4cbd/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.394965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb20171c1e24eb9af9a4f404f5d5a23, user_id=36236ddbf6c644f7a7e82fe8dffb4cbd, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.398271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.411766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 894/1790] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:41 2022] PUT /identity/v3/projects/9eb20171c1e24eb9af9a4f404f5d5a23/users/36236ddbf6c644f7a7e82fe8dffb4cbd/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:41.424355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.424704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.425193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.458029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] MFA Rules not processed for user `36236ddbf6c644f7a7e82fe8dffb4cbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:41.459413 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.535685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 897/1791] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:41.548036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.549833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.612956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fca606-d9a1-42a1-a103-4dc9b1d754ca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.617002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.617605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.617886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.618160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.626370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.627805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3c86e3-48d6-4985-96af-378d33e6339f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.637016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 895/1792] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:12:41.659369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.722340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e26a88-e36e-4578-9441-06512fa49e41 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.727993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.728618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.728900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.729202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] PATH_INFO: `/v3/users/36236ddbf6c644f7a7e82fe8dffb4cbd` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.730328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=36236ddbf6c644f7a7e82fe8dffb4cbd)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.732275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:41.777759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36236ddbf6c644f7a7e82fe8dffb4cbd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:41.784562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36236ddbf6c644f7a7e82fe8dffb4cbd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:41.791567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36236ddbf6c644f7a7e82fe8dffb4cbd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:41.792246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36236ddbf6c644f7a7e82fe8dffb4cbd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:41.792871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36236ddbf6c644f7a7e82fe8dffb4cbd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:41.810163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 898/1793] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:41 2022] DELETE /identity/v3/users/36236ddbf6c644f7a7e82fe8dffb4cbd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:41.814637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.814925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.815198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.834589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:41.837406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.882396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:41.895950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 896/1794] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:41.906836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:41.908251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:41.919401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:41.974073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636c376b-89f0-45b4-895c-53cbc3a44ea3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:41.978044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:41.978785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:41.979071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:41.979343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:41.988795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:41.990249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c00c0f2-5e41-42e9-9700-a88b26e6465c service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.001743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 899/1795] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:42.350566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.353058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:42.370277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2411b44f-478e-4986-87fc-b1531753b25e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.375072 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.375718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.376102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.376272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] PATH_INFO: `/v3/projects/9eb20171c1e24eb9af9a4f404f5d5a23` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.377419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb20171c1e24eb9af9a4f404f5d5a23)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.379419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.431492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb20171c1e24eb9af9a4f404f5d5a23', 'request_id': 'req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:42.437767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb20171c1e24eb9af9a4f404f5d5a23', 'request_id': 'req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:42.443287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb20171c1e24eb9af9a4f404f5d5a23', 'request_id': 'req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:42.456215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] The token cache is being invalidate because project 9eb20171c1e24eb9af9a4f404f5d5a23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:42.456497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:42.457113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8c7ea38c-1ab5-486e-a1fb-6dc7f28231a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:42.458610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 897/1796] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:42 2022] DELETE /identity/v3/projects/9eb20171c1e24eb9af9a4f404f5d5a23 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:42.463963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.464261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.464529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.484210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:42.485336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:42.546941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 900/1797] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:42.555387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.555789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.556109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.576940 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:42.578237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:42.587212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 898/1798] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:42.591420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.601484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f66e44-14c7-4dc5-95cb-48eb2f0a6838 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.606090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.606730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.607146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.607274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.607921 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:42.608236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.609952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.626225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 901/1799] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:42.630737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.640497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15355e9-2019-4330-a42b-87914df4f82f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.645306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.645918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.653038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.653356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.654025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.655943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.684697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 899/1800] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:42 2022] POST /identity/v3/projects => generated 365 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:42.690163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.704422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd0bec8-d468-42ea-b7ce-1fbe3f24a72c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.774145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.774788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.775104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.775380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.776016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.777948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.798929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 902/1801] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:42 2022] POST /identity/v3/users => generated 283 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:42.803067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.813436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab15a705-d1d3-478f-8900-ae62190cae15 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.817948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.818573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.818851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.819611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.820244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.821820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.831668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 900/1802] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:42.836400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.846554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61420d6-8439-4a2b-a184-ad9fb499129d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.851459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.852046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.852327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.852632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.853266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.854934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7f2ac7-970e-4341-8075-9647776ccb29 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.865392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 903/1803] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:42.869561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:42.879363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6f712-57f4-4b92-b0a4-3ee09f66e016 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:42.884493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:42.885131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.885414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.885692 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] PATH_INFO: `/v3/projects/c9b57afa51f94c04bbd45a29757b2e49/users/6905734d37ad4019ad224e2cb5735e87/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.900421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c9b57afa51f94c04bbd45a29757b2e49, user_id=6905734d37ad4019ad224e2cb5735e87, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:42.903089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:42.916004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 901/1804] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:42 2022] PUT /identity/v3/projects/c9b57afa51f94c04bbd45a29757b2e49/users/6905734d37ad4019ad224e2cb5735e87/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:42.927076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:42.927360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:42.927634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:42.957858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] MFA Rules not processed for user `6905734d37ad4019ad224e2cb5735e87`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:42.959124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:42.964071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:43.024871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 904/1805] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:43.036208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:43.038017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:43.095805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9a8cdb-65d7-4a1e-b480-58f5a9797a1f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:43.099539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:43.100154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:43.100433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:43.100744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:43.109216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:43.110656 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383ef3b-9ee9-4aad-a557-6238d61c532f service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:43.120709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 902/1806] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:43.801577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:43.803075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:43.871203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27090bfd-17e2-465e-a5bc-65a18b905027 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:43.875720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:43.876346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:43.876625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:43.876924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] PATH_INFO: `/v3/users/e609856fc9074514a6aeac24a3a6fe19` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:43.878039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e609856fc9074514a6aeac24a3a6fe19)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:43.879983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:43.925368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e609856fc9074514a6aeac24a3a6fe19'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:43.932620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e609856fc9074514a6aeac24a3a6fe19'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:43.939135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e609856fc9074514a6aeac24a3a6fe19'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:43.940011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e609856fc9074514a6aeac24a3a6fe19'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:43.940623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e609856fc9074514a6aeac24a3a6fe19'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:43.958036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 905/1807] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:43 2022] DELETE /identity/v3/users/e609856fc9074514a6aeac24a3a6fe19 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:43.962759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:43.963211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:43.963349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:43.981917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:43.983150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:43.992613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:44.044366 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 903/1808] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:44.054961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.056458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:44.087815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:44.116546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9ca744-3426-4aee-add1-0a7b66b260f1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.120527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.121128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.121409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.121707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.131017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.132629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9af2bc9-41f8-42f4-9d50-80bde9598ca3 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.142153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 906/1809] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:44.420073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.423054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:44.444041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2584c351-8a52-44d2-9c9b-baa5b73f270f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.449119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.450305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.450645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.450931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] PATH_INFO: `/v3/projects/1bcca70f71ad4565b50ae565c4f69195` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.452169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bcca70f71ad4565b50ae565c4f69195)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.454094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.512080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bcca70f71ad4565b50ae565c4f69195', 'request_id': 'req-a84ae29c-ab23-4905-8cd5-af92e445b807'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:44.519789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bcca70f71ad4565b50ae565c4f69195', 'request_id': 'req-a84ae29c-ab23-4905-8cd5-af92e445b807'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:44.531045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bcca70f71ad4565b50ae565c4f69195', 'request_id': 'req-a84ae29c-ab23-4905-8cd5-af92e445b807'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:44.546567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] The token cache is being invalidate because project 1bcca70f71ad4565b50ae565c4f69195 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:44.546933 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:44.547624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a84ae29c-ab23-4905-8cd5-af92e445b807 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:44.549195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 904/1810] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:44 2022] DELETE /identity/v3/projects/1bcca70f71ad4565b50ae565c4f69195 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:44.554869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.555182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.555512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.576857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:44.578105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:44.650576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 907/1811] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:44.658519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.658808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.659701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.699573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:44.700835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:44.714725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 905/1812] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:44.715763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.728665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126bdf48-7c6c-44c5-b713-06f4cb2380e7 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.734349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.735181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.735311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.735518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.736662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:44.736662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.738042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.752072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 908/1813] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:12:44.756881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.770253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63530ce0-a1be-48f2-81bd-d5a44fb06d83 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.777143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.777744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.778223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.778559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.779317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.781143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.814647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 906/1814] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:44 2022] POST /identity/v3/projects => generated 371 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:44.815857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.827181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bbeb2a-19e5-4f0e-a8a0-1b8434fe81fa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.911160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.912186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.913594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.914038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.914810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.918039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.948295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 909/1815] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:44 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:44.953667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.965329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a809891b-1a79-4e76-baf7-c2cc7f06267a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:44.972060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:44.972994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:44.973448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:44.974029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:44.974699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:44.977675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:44.999432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:44.999899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 907/1816] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:45.010984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9188e87-72ad-4dc8-8ef9-d7cd00cefd66 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:45.022984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:45.023933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.024275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.024624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.025403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:45.027317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71c8e90-7a58-492a-a120-627f8d96f62b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:45.067106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 910/1817] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:44 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:12:45.070461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:45.084992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9cadde-9ff8-4fdc-999d-8309f3b7dd34 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:45.098234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:45.098957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.099236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.099512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] PATH_INFO: `/v3/projects/ce2312616f314e8a85f755e7389d5f7e/users/4ae2865586c6471f8c456f27a961925c/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.125234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2312616f314e8a85f755e7389d5f7e, user_id=4ae2865586c6471f8c456f27a961925c, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:45.132641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:45.154499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 908/1818] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:45 2022] PUT /identity/v3/projects/ce2312616f314e8a85f755e7389d5f7e/users/4ae2865586c6471f8c456f27a961925c/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:45.161417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.161784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.162129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.193852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] MFA Rules not processed for user `4ae2865586c6471f8c456f27a961925c`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:45.195938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:45.222110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:45.266867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 911/1819] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:45 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:45.279995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:45.282240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:45.324396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:45.343312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de997ec8-e285-4663-bd9f-79749088a655 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:45.347053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:45.347666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.347952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.348232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.356619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:45.358032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3626b22-8475-4a98-b0db-5f9c396e4a22 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:45.367599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 909/1820] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:45 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:45.393404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:45.457006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b03842-5a1f-40f2-8812-9696233ca826 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:45.461663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:45.462288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.462604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.462883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] PATH_INFO: `/v3/users/4ae2865586c6471f8c456f27a961925c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.464063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ae2865586c6471f8c456f27a961925c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:45.466119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:45.514637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ae2865586c6471f8c456f27a961925c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:45.521373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ae2865586c6471f8c456f27a961925c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:45.528358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ae2865586c6471f8c456f27a961925c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:45.529271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ae2865586c6471f8c456f27a961925c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:45.529814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ae2865586c6471f8c456f27a961925c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:45.548878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 912/1821] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:45 2022] DELETE /identity/v3/users/4ae2865586c6471f8c456f27a961925c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:45.554839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.555133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.555428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.575733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:45.577055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:45.640681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 910/1822] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:45.650629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:45.652070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:45.706733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:45.720149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8275237d-1cb9-4de6-a0ae-f20e1a0d5515 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:45.723848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:45.724494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:45.724774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:45.725053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:45.735689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:45.737459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994c8bd-37b8-4f63-8010-569880882ad5 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:45.748836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 913/1823] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:12:46.041201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.043547 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:46.059799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cbd6c1-3dda-49fa-a8fd-ad43d525a073 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.064277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.067154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.067552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.068193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] PATH_INFO: `/v3/projects/ce2312616f314e8a85f755e7389d5f7e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.070284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2312616f314e8a85f755e7389d5f7e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.082949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.133178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2312616f314e8a85f755e7389d5f7e', 'request_id': 'req-aebb81e0-e239-40f9-bec9-b2b606690ae8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:46.140344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2312616f314e8a85f755e7389d5f7e', 'request_id': 'req-aebb81e0-e239-40f9-bec9-b2b606690ae8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:46.146767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2312616f314e8a85f755e7389d5f7e', 'request_id': 'req-aebb81e0-e239-40f9-bec9-b2b606690ae8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:46.164531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] The token cache is being invalidate because project ce2312616f314e8a85f755e7389d5f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:46.164934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:46.165997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-aebb81e0-e239-40f9-bec9-b2b606690ae8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:46.170969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 911/1824] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:46 2022] DELETE /identity/v3/projects/ce2312616f314e8a85f755e7389d5f7e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:46.173387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.173387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.173843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.196208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:46.197537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:46.276329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 914/1825] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:46.285835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.286144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.286734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.316269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:46.317719 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:46.336084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 912/1826] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:46.341900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.359382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f13894b-285b-4006-a253-a813b430e2a6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.365449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.367405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.367737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.368006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.368767 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:46.369108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.370914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.390597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 915/1827] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:46.392253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.419790 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f75060-cb4c-488e-ad2f-4b854ba8b297 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.425810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.426698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.427035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.427434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.428931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.431553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.479269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 913/1828] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:46 2022] POST /identity/v3/projects => generated 389 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:46.483237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.495689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548a055f-69d9-45c2-b0a6-654d3ea2ac84 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.577343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.577888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.578197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.578432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.579106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.581082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.616459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 916/1829] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:46 2022] POST /identity/v3/users => generated 295 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:46.619737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.633700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4908501a-dcd3-4edf-8395-df6fd8d7dce2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.641315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.641925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.642213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.642502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.643155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.645123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.660682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 914/1830] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:46.664510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.691532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f449a8a-d83d-4acb-b6c4-546780845cb0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.701092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.701761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.702183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.704584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.704584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.705108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcac4a76-36ad-4c35-b987-1afdab0f1076 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.716732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 917/1831] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:46 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:46.723642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.736489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbe092c-53dd-4511-9db3-b16a7ca81e81 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:46.742396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:46.743112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.743459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.743739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] PATH_INFO: `/v3/projects/719e994dee43405e9a48fde0a2f619c7/users/30d9392e4e444dfebe62a44cb0d06814/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.762974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=719e994dee43405e9a48fde0a2f619c7, user_id=30d9392e4e444dfebe62a44cb0d06814, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:46.766245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:46.776103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:46.783730 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 915/1832] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:46 2022] PUT /identity/v3/projects/719e994dee43405e9a48fde0a2f619c7/users/30d9392e4e444dfebe62a44cb0d06814/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:46.795027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b1038-4101-473c-ab0c-c72c7a2c33be None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:46.796470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b1038-4101-473c-ab0c-c72c7a2c33be None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:46.796726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b1038-4101-473c-ab0c-c72c7a2c33be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:46.838786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1b0b1038-4101-473c-ab0c-c72c7a2c33be None None] MFA Rules not processed for user `30d9392e4e444dfebe62a44cb0d06814`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:46.840422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1b0b1038-4101-473c-ab0c-c72c7a2c33be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:46.905281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:46.938513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 918/1833] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:46.950336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:46.952985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:47.022612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d27b39c-d816-42c0-bfc3-9b5dd8182dc6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:47.028604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:47.029554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:47.029869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:47.030158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:47.041332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:47.043007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa4397b-7ca4-449d-b92e-04f3b403f8b1 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:47.053010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 916/1834] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:46 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:53.171152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:53.171632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:53.171729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:53.173166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 919/1835] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:12:53.191766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:53.192317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:53.192317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:53.212783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:53.213981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:53.288284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 917/1836] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:12:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:53.304881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:53.306721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:53.330956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa275bb-0fc0-4331-8e0a-8fb3118dd0ff None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:53.337368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:53.338166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:53.339130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:53.339130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:53.350528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:53.352763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:53.363364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 920/1837] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:53.547723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:53.549218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:53.617948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd432db-346f-4846-aa67-6bef169326f5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:53.622405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:53.623055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:53.623333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:53.623610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] PATH_INFO: `/v3/users/5495217023fc44eab2fa73552db6b3f2` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:53.624711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5495217023fc44eab2fa73552db6b3f2)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:53.626700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:53.668099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:53.675654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5495217023fc44eab2fa73552db6b3f2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:53.683519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5495217023fc44eab2fa73552db6b3f2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:53.684204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5495217023fc44eab2fa73552db6b3f2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:53.691876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5495217023fc44eab2fa73552db6b3f2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:53.692797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-6da42c02-fdd4-4fcb-88c2-6f82f0372ba1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5495217023fc44eab2fa73552db6b3f2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:53.713025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 918/1838] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:12:53 2022] DELETE /identity/v3/users/5495217023fc44eab2fa73552db6b3f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:53.884820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:53.903109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e58452-c801-49e4-bd20-03f7ff20f030 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:53.964989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:53.965625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:53.965955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:53.966227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] PATH_INFO: `/v3/projects/d296608fc23c4667b5173c0f3fe1697e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:53.967373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d296608fc23c4667b5173c0f3fe1697e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:53.969396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.017578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd296608fc23c4667b5173c0f3fe1697e', 'request_id': 'req-63a951d3-3446-4db8-bd8d-94f0e443b994'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:54.025980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd296608fc23c4667b5173c0f3fe1697e', 'request_id': 'req-63a951d3-3446-4db8-bd8d-94f0e443b994'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:54.031879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd296608fc23c4667b5173c0f3fe1697e', 'request_id': 'req-63a951d3-3446-4db8-bd8d-94f0e443b994'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:54.052224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] The token cache is being invalidate because project d296608fc23c4667b5173c0f3fe1697e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:12:54.052669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:54.053848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-63a951d3-3446-4db8-bd8d-94f0e443b994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:12:54.059845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 921/1839] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:12:53 2022] DELETE /identity/v3/projects/d296608fc23c4667b5173c0f3fe1697e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:12:54.062200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.062844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.063505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.083997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:54.086049 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:54.147980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 919/1840] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:54.155819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.156176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.156520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.176308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:54.177536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:54.186938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 922/1841] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:54.190896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.201272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fcf78-35b2-4cb7-b23a-b38af5473308 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.205809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.206425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.206717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.206986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.207628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:12:54.207927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.209659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.220961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 920/1842] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:12:54.229539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.238698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3e4164-8e09-4cfd-b327-082884f2c974 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.243806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.244392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.244672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.244945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.245577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.247507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.275069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 923/1843] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:12:54 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:12:54.279308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.290906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a28831-6b8b-480d-b1b1-f6f9e050bb3e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.347207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.347824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.348129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.348363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.348980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.351556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.379346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 921/1844] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:12:54 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:12:54.380737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.392202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eefc3e7-4a36-4f13-bfb5-87f6bc270f07 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.401115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.401792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.402198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.402503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.403151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.404971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.417008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 924/1845] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:54.421237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.431958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6158d-d9cf-4667-ba7a-6fd62d15501c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.436750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.437289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.437565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.437837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.438554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.440097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a11fe3-73b5-440c-8dc6-7dc541b7b7e1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.452644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 922/1846] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:12:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:12:54.458139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.472126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eaa528-d504-47c9-862c-30436845b432 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.478618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.479581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.480076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.480431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] PATH_INFO: `/v3/projects/eab81e47f8d6426fbd5b50d1eb7fc14a/users/66975cf6d8fa417e8df429d4c89ef910/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.497380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab81e47f8d6426fbd5b50d1eb7fc14a, user_id=66975cf6d8fa417e8df429d4c89ef910, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.500289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.504412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:54.522300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 925/1847] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:12:54 2022] PUT /identity/v3/projects/eab81e47f8d6426fbd5b50d1eb7fc14a/users/66975cf6d8fa417e8df429d4c89ef910/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:12:54.527458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225be3-7e98-4c27-ad26-c3373d90da46 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.527746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225be3-7e98-4c27-ad26-c3373d90da46 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.528036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93225be3-7e98-4c27-ad26-c3373d90da46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.547902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93225be3-7e98-4c27-ad26-c3373d90da46 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:54.549115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93225be3-7e98-4c27-ad26-c3373d90da46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:54.565609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:12:54.608311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 923/1848] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:12:54.618699 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:54.620436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:54.679660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ca3063-1d4a-44b9-99b6-360c401e73b7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:54.683302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:54.683922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:54.684204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:54.684478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:54.694044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:54.695605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:54.705864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 926/1849] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:12:58.996676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931758-a388-49d3-9749-0887d9f8dbb2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:58.997157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931758-a388-49d3-9749-0887d9f8dbb2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:58.997374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af931758-a388-49d3-9749-0887d9f8dbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:59.052153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af931758-a388-49d3-9749-0887d9f8dbb2 None None] MFA Rules not processed for user `66975cf6d8fa417e8df429d4c89ef910`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:12:59.054706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af931758-a388-49d3-9749-0887d9f8dbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:59.134225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 924/1850] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:12:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:12:59.149098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:12:59.150733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:12:59.231477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2737f9c-0bd7-436c-934a-fdd5eec87e5e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:12:59.237140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:12:59.237834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:12:59.238116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:12:59.238396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:12:59.248011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:12:59.249639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4756d61c-80fb-4550-ae0b-c7c56c8e3ff9 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:12:59.260644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 927/1851] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:12:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:13:00.916307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:00.916826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:00.917037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:00.937388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] MFA Rules not processed for user `66975cf6d8fa417e8df429d4c89ef910`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:00.938648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:00.947195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 925/1852] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:09.255308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=62 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:10.931605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=52 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:15.896027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7d27-d383-46f7-a004-315c265bbf6b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:15.896924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7d27-d383-46f7-a004-315c265bbf6b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:15.897317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7d27-d383-46f7-a004-315c265bbf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:15.925507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-848f7d27-d383-46f7-a004-315c265bbf6b None None] MFA Rules not processed for user `04ee886aeea740428a58a513600fc9b8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:15.926961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-848f7d27-d383-46f7-a004-315c265bbf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:15.994249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 928/1853] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:16.088526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:16.090017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:16.112758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0091b-997c-44a3-ba02-fd43893434ec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:16.140282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:16.141023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:16.141306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:16.141582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] PATH_INFO: `/v3/users/30d9392e4e444dfebe62a44cb0d06814` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:16.143044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30d9392e4e444dfebe62a44cb0d06814)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:16.145939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:16.196809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30d9392e4e444dfebe62a44cb0d06814'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:16.203868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30d9392e4e444dfebe62a44cb0d06814'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:16.204565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30d9392e4e444dfebe62a44cb0d06814'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:16.216328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30d9392e4e444dfebe62a44cb0d06814'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:16.219149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30d9392e4e444dfebe62a44cb0d06814'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:16.239245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 926/1854] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:13:16 2022] DELETE /identity/v3/users/30d9392e4e444dfebe62a44cb0d06814 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:16.243741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:16.245152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:16.245438 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:16.276552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:16.278080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:16.351246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 929/1855] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:16.365459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:16.366908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:16.432556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fd8206-8c7d-456c-8cd1-a44f5b674556 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:16.437573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:16.438314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:16.438635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:16.439062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:16.457633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:16.459295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec682700-5468-47f0-97ef-dc50d15f692f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:16.470965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 927/1856] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:13:16.862656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:16.864785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:16.913068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94245d6-a28c-4e9a-8493-1802239e3796 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:16.918061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:16.919085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:16.919440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:16.919796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] PATH_INFO: `/v3/projects/719e994dee43405e9a48fde0a2f619c7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:16.921022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=719e994dee43405e9a48fde0a2f619c7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:16.923224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:16.998784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '719e994dee43405e9a48fde0a2f619c7', 'request_id': 'req-111effd7-fdad-419f-be45-a32d76ac96ec'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:17.007268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '719e994dee43405e9a48fde0a2f619c7', 'request_id': 'req-111effd7-fdad-419f-be45-a32d76ac96ec'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:17.014977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '719e994dee43405e9a48fde0a2f619c7', 'request_id': 'req-111effd7-fdad-419f-be45-a32d76ac96ec'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:17.040761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] The token cache is being invalidate because project 719e994dee43405e9a48fde0a2f619c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:13:17.041057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:17.041721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-111effd7-fdad-419f-be45-a32d76ac96ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:17.043970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 930/1857] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:13:16 2022] DELETE /identity/v3/projects/719e994dee43405e9a48fde0a2f619c7 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:17.049054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.049706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.049706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.070499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:17.071721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:17.140342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 928/1858] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:13:17.153365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.154169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.154549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.186483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:17.187176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:17.202004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 931/1859] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:17.206194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.217626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c332926a-7ac2-4718-8518-0b11d43a9da9 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.222317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.223014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.223299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.223575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.224229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:13:17.224539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.226152 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.241869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 929/1860] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:13:17.243281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.252764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd07645-a062-4df5-8aaa-9064937f2c88 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.258124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.258824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.259109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.259379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.260016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.261905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.284446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:17.298662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 932/1861] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:13:17 2022] POST /identity/v3/projects => generated 377 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:13:17.301216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.307314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:17.312714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad0d33b-72d2-468b-b510-8af7d320ca71 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.377479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.378278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.378741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.379168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.380005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.382588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.411868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 930/1862] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:13:17 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:13:17.415415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.426034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd23adcd-9f4d-4c11-a25c-7dbc353ded0b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.430641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.431239 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.431522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.431802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.432435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.434035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.445219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 933/1863] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:13:17.449842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.460787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3eac1a-725d-4473-aa88-f64022328af2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.465356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.465957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.466237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.466629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.467263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.468863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1755f7ea-71c5-478c-934c-a9a6096fbcea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.480138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 931/1864] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:13:17.484360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.494490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8a49d1-0d60-465c-8495-4ff617b4d3e0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.499072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.499700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.499986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.500281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] PATH_INFO: `/v3/projects/10f27676e78f475fa9734f3d6f541da5/users/d8cd3210dbea46d6912c93c6d28356ae/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.515331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=10f27676e78f475fa9734f3d6f541da5, user_id=d8cd3210dbea46d6912c93c6d28356ae, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.517985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.533035 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 934/1865] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:13:17 2022] PUT /identity/v3/projects/10f27676e78f475fa9734f3d6f541da5/users/d8cd3210dbea46d6912c93c6d28356ae/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:17.538524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430118a4-3b85-45d8-a4f4-23d2af49afff None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.538871 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430118a4-3b85-45d8-a4f4-23d2af49afff None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.539150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430118a4-3b85-45d8-a4f4-23d2af49afff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.558022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-430118a4-3b85-45d8-a4f4-23d2af49afff None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:17.559267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-430118a4-3b85-45d8-a4f4-23d2af49afff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:17.617929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 932/1866] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:17.627672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:17.629982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:17.688352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21808d8b-6016-4eb7-8bf9-30cb84863b55 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:17.694068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:17.695002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:17.695480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:17.695757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:17.706009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:17.708065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:17.719455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 935/1867] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:13:22.972691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:22.973285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:22.973285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.011880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] MFA Rules not processed for user `d8cd3210dbea46d6912c93c6d28356ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:23.013109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:23.111091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 933/1868] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:13:23.119175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.126643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87881076-e412-4d0d-b10b-e4a2f22e0cfa service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.136271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.136271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.136271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.137430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.138174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.140099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.178814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 936/1869] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:13:23 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:13:23.183335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.196230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00359301-19c2-4d28-b0f5-0dbf92af6ca8 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.206643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:23.280949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.281745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.282010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.282292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.283746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.285136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.313791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 934/1870] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:13:23 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:13:23.318745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.331945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0256d48-6c2d-44c6-a5e6-39c56b75465a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.343145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.343963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.344343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.344748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.345614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.347963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.363973 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 937/1871] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:13:23.368538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.390396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc4a7b2-e06d-4fce-8f5f-09731192b2c2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.397440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.398315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.398781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.399351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] PATH_INFO: `/v3/projects/94ac7006ccce45c19464a0adfbe41af9/users/3e4d471ec3dc43e08197dab0c9ec8ca7/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.428958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ac7006ccce45c19464a0adfbe41af9, user_id=3e4d471ec3dc43e08197dab0c9ec8ca7, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.434175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.463830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 935/1872] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:13:23 2022] PUT /identity/v3/projects/94ac7006ccce45c19464a0adfbe41af9/users/3e4d471ec3dc43e08197dab0c9ec8ca7/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:23.465285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.482586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92f9f0a-6ba0-4125-be4e-9f6fe58e0a17 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.577407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.578174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.579456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.579998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.581378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.583937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db41c3f6-a765-4e90-9839-6c19555026d0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.601810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 938/1873] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:13:23.605786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:23.621345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eaf1d62-f4fd-4761-aa77-0878281f7a98 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:23.629296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:23.630137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:23.630570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:23.630956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] PATH_INFO: `/v3/projects/94ac7006ccce45c19464a0adfbe41af9/users/3e4d471ec3dc43e08197dab0c9ec8ca7/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:23.633917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ac7006ccce45c19464a0adfbe41af9, user_id=3e4d471ec3dc43e08197dab0c9ec8ca7, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:23.637535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:23.658913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 936/1874] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:13:23 2022] PUT /identity/v3/projects/94ac7006ccce45c19464a0adfbe41af9/users/3e4d471ec3dc43e08197dab0c9ec8ca7/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:29.082876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:29.084084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:29.084520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:29.139801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] MFA Rules not processed for user `3e4d471ec3dc43e08197dab0c9ec8ca7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:29.149143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:29.189663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:29.304052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 939/1875] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:29.331397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:29.333455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:29.442024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b157e672-0292-4021-af9d-c4bdd3ea0d30 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:29.448258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:29.449010 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:29.449336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:29.449616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:29.524722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:29.525987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f461b9e-74a2-4a8c-aef5-ab644713d8a3 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:29.531523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:29.550769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 937/1876] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:13:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:13:39.305074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=20 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:42.926165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:42.927701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:42.999410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a081592-711b-4892-8fcc-7453a2f23608 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:43.004539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:43.005214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.005517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.005829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] PATH_INFO: `/v3/users/6905734d37ad4019ad224e2cb5735e87` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.007091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6905734d37ad4019ad224e2cb5735e87)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:43.009075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:43.058214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6905734d37ad4019ad224e2cb5735e87'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.065434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6905734d37ad4019ad224e2cb5735e87'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.072327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6905734d37ad4019ad224e2cb5735e87'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.073259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6905734d37ad4019ad224e2cb5735e87'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.074022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6905734d37ad4019ad224e2cb5735e87'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.096180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 940/1877] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:13:42 2022] DELETE /identity/v3/users/6905734d37ad4019ad224e2cb5735e87 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:43.100941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.101499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.101594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.122082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:43.123605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:43.194106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 938/1878] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:43.207192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:43.208756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:43.280605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb5ef56-310d-4074-99d4-b71bd9e859ea admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:43.284502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:43.285115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.285394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.285677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.295109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:43.296566 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f320f-9a9e-4c4d-995c-a8aa04f5431f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:43.307226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 941/1879] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:13:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:13:43.580118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:43.581529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:43.593179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27caa33-4627-44b7-8992-c0020adc4b18 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:43.598779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:43.599511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.599824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.600077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] PATH_INFO: `/v3/projects/c9b57afa51f94c04bbd45a29757b2e49` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.601346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c9b57afa51f94c04bbd45a29757b2e49)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:43.603365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:43.651166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c9b57afa51f94c04bbd45a29757b2e49', 'request_id': 'req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.657880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c9b57afa51f94c04bbd45a29757b2e49', 'request_id': 'req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.663774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c9b57afa51f94c04bbd45a29757b2e49', 'request_id': 'req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.683893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] The token cache is being invalidate because project c9b57afa51f94c04bbd45a29757b2e49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:13:43.683893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.683893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a03d7ee0-e00b-45af-bde5-5e68bb6acf9c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:13:43.685105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 939/1880] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:13:43 2022] DELETE /identity/v3/projects/c9b57afa51f94c04bbd45a29757b2e49 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:43.689482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.690010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.690595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.723201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:43.724602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:43.745151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:43.792684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 942/1881] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:13:43.801603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.801985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.802323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.823092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:43.824313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:43.832947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 940/1882] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:43.837394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:43.847784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a52400-1edb-4cc2-a057-b9954b1f354c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:43.852783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:43.853551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.854033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.854310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.855243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:13:43.855699 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:43.857868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:43.875233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 943/1883] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:13:43.876430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:43.885882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df215ae-e868-4798-a677-2bed4e665ae7 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:43.891529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:43.892143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:43.892443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:43.892693 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:43.893339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:43.895278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:43.925763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 941/1884] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:13:43.930280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:43.942339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c15fc7-f485-43be-91f1-27c3d6170567 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:44.005004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:44.005630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.005917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.006178 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.006859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:44.008748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:44.032947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 944/1885] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:13:43 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:13:44.034101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:44.044646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6beec18c-ac08-4f68-bed4-7d111d5b374f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:44.049872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:44.050541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.050824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.051123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.051793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:44.053453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:44.063319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 942/1886] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:13:44.067261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:44.077168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e38695-8730-474f-a702-abafce3c4c5d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:44.082361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:44.082989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.083302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.083580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.084219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:44.085815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d760bf6e-6990-41be-a2f4-4f99fddaa889 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:44.097895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 945/1887] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:13:44.101978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:44.112201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891da55-dd7c-484c-b4ac-2667777c2d3c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:44.117082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:44.117739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.118025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.118302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] PATH_INFO: `/v3/projects/d79a221e6cff4d368a66b3d06a8f244f/users/2acf9cf2d9f842bab1d8d47fa7b98280/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.133987 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d79a221e6cff4d368a66b3d06a8f244f, user_id=2acf9cf2d9f842bab1d8d47fa7b98280, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:44.136744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:44.151349 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 943/1888] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:13:44 2022] PUT /identity/v3/projects/d79a221e6cff4d368a66b3d06a8f244f/users/2acf9cf2d9f842bab1d8d47fa7b98280/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:13:44.163427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3807f-19c2-4cfe-b5aa-113e83c49376 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.163712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3807f-19c2-4cfe-b5aa-113e83c49376 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.163985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b3807f-19c2-4cfe-b5aa-113e83c49376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.193091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-60b3807f-19c2-4cfe-b5aa-113e83c49376 None None] MFA Rules not processed for user `2acf9cf2d9f842bab1d8d47fa7b98280`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:44.194228 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-60b3807f-19c2-4cfe-b5aa-113e83c49376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:44.267840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 946/1889] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:44.292539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:44.295020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:44.321598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:44.362762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c347c45-9751-4acd-9432-9e711fa27c0d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:44.366570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:44.367214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:44.367622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:44.367895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:44.376278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:44.377752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1486609-c398-4019-910e-3a89cd509fd0 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:44.387093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 944/1890] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:13:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:13:48.099596 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:48.100140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:48.100140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:48.121963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] MFA Rules not processed for user `2335878ea1d045858b158f6b4175e896`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:48.123207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:48.135681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:13:48.189392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 947/1891] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:13:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:53.662850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:53.663322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:53.663523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:53.693199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:13:53.694402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:53.763316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 945/1892] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:13:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:13:53.815220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:13:53.816754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:13:53.924705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b5598-5ccb-4174-9115-b36603effb00 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:13:53.930852 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:13:53.932708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:13:53.933341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:13:53.933605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:13:53.944196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:13:53.945685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb82b1c-ee6d-4a97-a18e-c5bfa8b10f9c service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:13:53.958942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 948/1893] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:13:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:14:03.750422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=42 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:03.920899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:03.953598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=49 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:04.009587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b3be4b-5936-464b-8992-222757a0a392 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:04.014802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:04.015820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:04.016284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:04.016706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] PATH_INFO: `/v3/users/2acf9cf2d9f842bab1d8d47fa7b98280` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:04.018098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2acf9cf2d9f842bab1d8d47fa7b98280)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:04.020339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:04.085856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2acf9cf2d9f842bab1d8d47fa7b98280'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:04.112851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2acf9cf2d9f842bab1d8d47fa7b98280'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:04.113606 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2acf9cf2d9f842bab1d8d47fa7b98280'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:04.142764 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2acf9cf2d9f842bab1d8d47fa7b98280'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:04.144046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2acf9cf2d9f842bab1d8d47fa7b98280'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:04.191851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 946/1894] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:14:03 2022] DELETE /identity/v3/users/2acf9cf2d9f842bab1d8d47fa7b98280 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:04.196242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906a4c-6ace-4928-a505-ea1377c6153c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:04.196729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906a4c-6ace-4928-a505-ea1377c6153c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:04.197119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a906a4c-6ace-4928-a505-ea1377c6153c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:04.220806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0a906a4c-6ace-4928-a505-ea1377c6153c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:04.223099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0a906a4c-6ace-4928-a505-ea1377c6153c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:04.332620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 949/1895] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:04.343118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:04.345459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:04.450516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab5be0b-850e-4210-8e7d-8185a5968ed6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:04.457056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:04.457909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:04.458295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:04.458754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:04.472685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:04.475306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:04.500707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 947/1896] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:04.897779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576801b-0940-44a1-9e51-6e2888f2f897 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:04.898066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576801b-0940-44a1-9e51-6e2888f2f897 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:04.898342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a576801b-0940-44a1-9e51-6e2888f2f897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:04.919515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a576801b-0940-44a1-9e51-6e2888f2f897 None None] MFA Rules not processed for user `d8cd3210dbea46d6912c93c6d28356ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:04.920812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a576801b-0940-44a1-9e51-6e2888f2f897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:04.996266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 950/1897] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:05.086685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.088869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:05.124167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634b46aa-6255-4df2-b4ca-2fb7ebba4369 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.135369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.143720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.143720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.143720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] PATH_INFO: `/v3/projects/d79a221e6cff4d368a66b3d06a8f244f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.145455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d79a221e6cff4d368a66b3d06a8f244f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.147371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.186075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:05.244645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd79a221e6cff4d368a66b3d06a8f244f', 'request_id': 'req-8fa83bbc-94c2-45f0-9226-654ecbc8929d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:05.254136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd79a221e6cff4d368a66b3d06a8f244f', 'request_id': 'req-8fa83bbc-94c2-45f0-9226-654ecbc8929d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:05.263277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd79a221e6cff4d368a66b3d06a8f244f', 'request_id': 'req-8fa83bbc-94c2-45f0-9226-654ecbc8929d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:05.278705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] The token cache is being invalidate because project d79a221e6cff4d368a66b3d06a8f244f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:14:05.279308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:05.280626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8fa83bbc-94c2-45f0-9226-654ecbc8929d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:05.282917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 948/1898] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:14:05 2022] DELETE /identity/v3/projects/d79a221e6cff4d368a66b3d06a8f244f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:05.288045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.288464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.288763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.311534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:05.313775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:05.392375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 951/1899] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:05.403508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.404195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.404808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.436592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:05.438659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:05.452291 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 949/1900] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:05.457000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.467510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e633be-37f4-45eb-9858-6d6d92a5c20f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.473984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.475069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.475367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.475761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.476723 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:14:05.477044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.479361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.499205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 952/1901] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:14:05.501194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.513254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8568fdd3-90aa-4cc5-bffb-7664f5a78ecd None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.520255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.520978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.521262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.521532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.522176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.525408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.569083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 950/1902] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:14:05 2022] POST /identity/v3/projects => generated 373 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:14:05.570575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.584935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e9e6d7-a97b-482d-811a-95f1bc15e984 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.637814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:05.670942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.672180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.672481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.672764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.673419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.675450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.710891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 953/1903] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:14:05 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:14:05.712130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.725991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3697d91-b16a-4079-b6ba-191ecf333602 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.741698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.742760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.743280 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.743618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.744403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.746993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.763204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 951/1904] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:05.764088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.790724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc59336-c970-4390-97d4-e3f300a8b4bc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.798216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.798988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.799373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.799778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.800735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.802981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486fe9ea-f228-4e8d-9dd7-86312b9bc1da admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.820809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 954/1905] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:05 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:05.822012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:05.832304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691b1ddc-a9bb-4834-90ee-c91ccfa5ce83 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:05.838745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:05.839416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.839697 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.839974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] PATH_INFO: `/v3/projects/d7bb15a157314fb095d2333f64ab0c83/users/102283eee43142f29645e3faa6501891/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.855843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7bb15a157314fb095d2333f64ab0c83, user_id=102283eee43142f29645e3faa6501891, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:05.858565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:05.878694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 952/1906] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:14:05 2022] PUT /identity/v3/projects/d7bb15a157314fb095d2333f64ab0c83/users/102283eee43142f29645e3faa6501891/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:05.880487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7b69-f282-4b16-9162-5778e8feabc3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:05.880799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7b69-f282-4b16-9162-5778e8feabc3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:05.881173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99a7b69-f282-4b16-9162-5778e8feabc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:05.905809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f99a7b69-f282-4b16-9162-5778e8feabc3 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:05.907109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f99a7b69-f282-4b16-9162-5778e8feabc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:06.033983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 955/1907] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:06.041363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:06.043025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:06.113157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74200a6-c258-4499-a26a-85cf6a022938 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:06.117730 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:06.118420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:06.118741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:06.119151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:06.131914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:06.135076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:06.147568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 953/1908] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:12.436088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:12.436804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:12.436804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:12.507136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] MFA Rules not processed for user `102283eee43142f29645e3faa6501891`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:12.508631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:12.658725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:12.667144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 956/1909] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:12.676380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045b0fc-6297-49c2-97d7-cd587bd1abea service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:12.708057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:12.709044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:12.709593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:12.710609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:12.711924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:12.715959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:12.787714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:12.798768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 954/1910] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:14:12 2022] POST /identity/v3/projects => generated 373 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:14:12.810748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe17b09-9106-4a45-82ff-e5e066174fc6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:12.891585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:12.958317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:12.960372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:12.960862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:12.961548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:12.962795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:12.966166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:13.007131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 957/1911] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:14:12 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:14:13.008437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:13.023855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1990776-5724-486a-b1a7-cc661d3d17c8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:13.030623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:13.031753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:13.032143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:13.032659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:13.033466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:13.035645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:13.063625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:13.072011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 955/1912] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:13.080527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0410a93-a582-44a3-8e5c-453e7f4d87e1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:13.092283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:13.094295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:13.095074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:13.095362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] PATH_INFO: `/v3/projects/e2c335c445c94b8bbec595e062c254a6/users/a7e3e6ba208d40b28e6a44a7f61fb1cf/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:13.136591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c335c445c94b8bbec595e062c254a6, user_id=a7e3e6ba208d40b28e6a44a7f61fb1cf, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:13.139336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:13.168741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 958/1913] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:14:13 2022] PUT /identity/v3/projects/e2c335c445c94b8bbec595e062c254a6/users/a7e3e6ba208d40b28e6a44a7f61fb1cf/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:13.173388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:13.190971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0383b8a-9004-41e9-8df4-2dd3017dd0da admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:13.221346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:13.345351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:13.346231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:13.360174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:13.370080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:13.371823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:13.373519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fa9760-6a4f-4c45-bff5-d08715b9b4dc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:13.412671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 956/1914] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:13 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:13.415691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:13.443510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a348fab-4246-49b9-b881-d9347bfc632c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:13.460307 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:13.462930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:13.463239 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:13.463518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] PATH_INFO: `/v3/projects/e2c335c445c94b8bbec595e062c254a6/users/a7e3e6ba208d40b28e6a44a7f61fb1cf/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:13.467320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2c335c445c94b8bbec595e062c254a6, user_id=a7e3e6ba208d40b28e6a44a7f61fb1cf, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:13.471013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:13.492987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 959/1915] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:14:13 2022] PUT /identity/v3/projects/e2c335c445c94b8bbec595e062c254a6/users/a7e3e6ba208d40b28e6a44a7f61fb1cf/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:17.882533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a140-028f-483f-804c-c09839846676 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:17.883427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a140-028f-483f-804c-c09839846676 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:17.883857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2624a140-028f-483f-804c-c09839846676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:17.913601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2624a140-028f-483f-804c-c09839846676 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:17.915641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2624a140-028f-483f-804c-c09839846676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:18.054782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 957/1916] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:14:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:18.076157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:18.078327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:18.260399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197268a3-6ecc-4a11-a62e-d1ff7ddc1d19 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:18.280671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:18.281447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:18.281734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:18.282196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:18.299613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:18.301322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:18.315975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 960/1917] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:14:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:14:19.588458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:19.588973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:19.589066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:19.619382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] MFA Rules not processed for user `a7e3e6ba208d40b28e6a44a7f61fb1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:19.620703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:19.693700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 958/1918] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:19.707892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:19.709456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:19.783470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c8bbd4-776d-41ec-a54b-7ecc75c0bc93 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:19.790033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:19.790751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:19.791036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:19.791307 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:19.811651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:19.850568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:19.852161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0680d-2db7-4690-9de1-c456408cd9ab service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:19.861044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 961/1919] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:20.797442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:20.799076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:20.857441 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:20.886223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d11b8d-bb1c-40df-a7c8-964321b48d77 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:20.891640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:20.892294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:20.892614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:20.893552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] PATH_INFO: `/v3/users/04ee886aeea740428a58a513600fc9b8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:20.894801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04ee886aeea740428a58a513600fc9b8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:20.896755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:20.957301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04ee886aeea740428a58a513600fc9b8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:20.965648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04ee886aeea740428a58a513600fc9b8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:20.967488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04ee886aeea740428a58a513600fc9b8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:20.974961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04ee886aeea740428a58a513600fc9b8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:20.976535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04ee886aeea740428a58a513600fc9b8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.001632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 959/1920] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:14:20 2022] DELETE /identity/v3/users/04ee886aeea740428a58a513600fc9b8 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:21.255037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67249744-6884-4656-a3fe-4b7557360f1c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.255555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67249744-6884-4656-a3fe-4b7557360f1c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.255649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67249744-6884-4656-a3fe-4b7557360f1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.280082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-67249744-6884-4656-a3fe-4b7557360f1c None None] MFA Rules not processed for user `102283eee43142f29645e3faa6501891`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:21.281867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-67249744-6884-4656-a3fe-4b7557360f1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:21.289850 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:21.291747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:21.364441 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 962/1921] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:21.379836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d87bd-2ab3-47ad-aa45-c18b2f5bc136 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:21.385877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:21.386557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.386838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.387112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] PATH_INFO: `/v3/projects/e84f012d59214a6bb6e7f5490bfb2bfc` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.388270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e84f012d59214a6bb6e7f5490bfb2bfc)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:21.390995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:21.484552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e84f012d59214a6bb6e7f5490bfb2bfc', 'request_id': 'req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.496695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e84f012d59214a6bb6e7f5490bfb2bfc', 'request_id': 'req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.504343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:21.507747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e84f012d59214a6bb6e7f5490bfb2bfc', 'request_id': 'req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.525437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] The token cache is being invalidate because project e84f012d59214a6bb6e7f5490bfb2bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:14:21.525733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.526627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9dc4dd45-6236-45cf-a53d-5d4ce1cf31ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:21.528595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 960/1922] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:14:21 2022] DELETE /identity/v3/projects/e84f012d59214a6bb6e7f5490bfb2bfc => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:14:21.535115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.535516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.535896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.560052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:21.562186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:21.636264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 963/1923] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:21.643775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.644620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.644898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.692062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:21.693830 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:21.712344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 961/1924] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:14:21.713592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:21.727252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b69af0-6583-41ed-b004-77abaedb79d6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:21.734423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:21.735812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.736289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.736717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.737783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:14:21.738265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:21.740141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:21.764703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 964/1925] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:14:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:14:21.768546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:21.779990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680a1ed9-d53a-4c22-85f4-1de261910cb5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:21.789385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:21.790252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.790737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.791190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.792128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:21.794720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:21.838757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 962/1926] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:14:21 2022] POST /identity/v3/projects => generated 385 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:14:21.843051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:21.856660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e279591e-81bf-43de-89f5-a74be290176d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:21.950513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:21.963092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:21.964383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:21.964753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:21.965218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:21.966383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:21.971133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:22.024581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 965/1927] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:14:21 2022] POST /identity/v3/users => generated 293 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:14:22.026085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:22.054477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7a2f4-fcc7-4ad9-b34e-ad922bd95905 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:22.061098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:22.062293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:22.062640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:22.062918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:22.063569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:22.065709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:22.078121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 963/1928] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:22.082273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:22.092964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355c41d-f347-41cc-adf3-4744aa23c9be admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:22.097976 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:22.098902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:22.098902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:22.099334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:22.099987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:22.101593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf42ba7-b237-4fa0-9bfa-716d4a173ed4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:22.113991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 966/1929] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:14:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:14:22.118149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:22.129355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff68bf-3082-4caf-b319-28d780a7cad0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:22.136745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:22.137465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:22.137748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:22.138024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] PATH_INFO: `/v3/projects/e7682569529c408fb332908d59519f91/users/df81d7d1abab4e7f9d0b8423c60946e7/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:22.160913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7682569529c408fb332908d59519f91, user_id=df81d7d1abab4e7f9d0b8423c60946e7, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:22.163905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:22.178006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 964/1930] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:14:22 2022] PUT /identity/v3/projects/e7682569529c408fb332908d59519f91/users/df81d7d1abab4e7f9d0b8423c60946e7/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:14:22.189776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:22.190093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:22.190369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:22.220267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] MFA Rules not processed for user `df81d7d1abab4e7f9d0b8423c60946e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:22.221512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:22.297205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 967/1931] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:22.309143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:22.311642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:22.372206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fc7c27-2bf4-4dc0-b255-f4b23a1954a4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:22.377479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:22.378176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:22.378568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:22.378912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:22.389214 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:22.390887 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80a0b7c-a397-4502-ba08-112f37e59801 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:22.404659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 965/1932] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:22 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:25.061401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:25.128021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb18189-8a6d-420a-9220-be7af8ce19d2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:25.132437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:25.133091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:25.133370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:25.133668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] PATH_INFO: `/v3/users/df81d7d1abab4e7f9d0b8423c60946e7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:25.134849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df81d7d1abab4e7f9d0b8423c60946e7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:25.136750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:25.196688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df81d7d1abab4e7f9d0b8423c60946e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.204129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df81d7d1abab4e7f9d0b8423c60946e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.210761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df81d7d1abab4e7f9d0b8423c60946e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.211389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df81d7d1abab4e7f9d0b8423c60946e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.212001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df81d7d1abab4e7f9d0b8423c60946e7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.213168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:25.235494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 968/1933] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:14:25 2022] DELETE /identity/v3/users/df81d7d1abab4e7f9d0b8423c60946e7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:25.239940 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:25.240476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:25.240476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:25.260051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:25.261253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:25.322144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 966/1934] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:25.333172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:25.334650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:25.401593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdb75f1-d2f9-4474-b95d-5c5372149728 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:25.406196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:25.406855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:25.407138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:25.407420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:25.417450 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:25.418919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:25.430102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 969/1935] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:25.754127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:25.755584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:25.768503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1141d00e-17f2-4ebb-af66-cca06501a93e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:25.773902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:25.774769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:25.775151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:25.775407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] PATH_INFO: `/v3/projects/e7682569529c408fb332908d59519f91` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:25.783174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7682569529c408fb332908d59519f91)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:25.790928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:25.815660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:25.841076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7682569529c408fb332908d59519f91', 'request_id': 'req-1d154697-d070-4f7e-b400-72e145ed8d3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.847594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7682569529c408fb332908d59519f91', 'request_id': 'req-1d154697-d070-4f7e-b400-72e145ed8d3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.855619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7682569529c408fb332908d59519f91', 'request_id': 'req-1d154697-d070-4f7e-b400-72e145ed8d3e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.868590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] The token cache is being invalidate because project e7682569529c408fb332908d59519f91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:14:25.868883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.869499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d154697-d070-4f7e-b400-72e145ed8d3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:14:25.871060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 967/1936] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:14:25 2022] DELETE /identity/v3/projects/e7682569529c408fb332908d59519f91 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:14:29.461925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:29.463433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:29.475306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc19bca9-17ce-4df5-b2fe-76df3e4903b8 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:29.530097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:29.530749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:29.531029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:29.531309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:29.533027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:29.597418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:29.598953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775b21db-5bdb-4dc1-b00a-7c59d7b7c5c4 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:29.609146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 970/1937] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:35.865225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:39.604121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:48.953273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17183-5d66-4ed9-b3cb-32f0e4c2f7de None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:48.953707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17183-5d66-4ed9-b3cb-32f0e4c2f7de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:48.953947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee17183-5d66-4ed9-b3cb-32f0e4c2f7de None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:48.956116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 968/1938] 10.222.0.23 () {54 vars in 961 bytes} [Sun Oct 23 02:14:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:14:48.962509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd63d-d047-4291-8e59-aac2eae4aa85 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:48.963043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd63d-d047-4291-8e59-aac2eae4aa85 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:48.963043 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556fd63d-d047-4291-8e59-aac2eae4aa85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:48.983508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-556fd63d-d047-4291-8e59-aac2eae4aa85 None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:14:48.984748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-556fd63d-d047-4291-8e59-aac2eae4aa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:49.055215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 971/1939] 10.222.0.23 () {58 vars in 1058 bytes} [Sun Oct 23 02:14:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:14:49.063646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee17183-5d66-4ed9-b3cb-32f0e4c2f7de None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:14:49.078658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee17183-5d66-4ed9-b3cb-32f0e4c2f7de None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:14:49.083757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:14:49.084440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:14:49.084729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:14:49.085002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:14:49.086761 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:14:49.099038 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:14:49.100614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:14:49.111401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 969/1940] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:14:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:14:59.045629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=29 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:14:59.106596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:15:05.900613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:05.901120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:05.901213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:05.920677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:05.921855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:05.974854 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 972/1941] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:15:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:15:05.997460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:05.998913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:06.066221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de43718-1ff4-45d5-871d-2632e4fa7b32 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:06.071164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:06.071771 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:06.072081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:06.072354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:06.080577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:06.082183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ef4595-bdae-4b75-884e-95fb774895a2 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:06.090085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 970/1942] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:15:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:15:08.317474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:08.319063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:08.331730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375dbac9-d581-444c-840a-5e74df9215ed None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:08.335483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:08.336181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:08.336532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:08.336845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:08.339325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:08.412761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:08.414272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e2bcb2-ba73-4150-9ec6-eb6aed25f891 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:08.426146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 973/1943] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:15:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:15:16.086828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=32 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:15:18.420632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=55 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:15:20.952108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:20.952932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:20.953881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:20.955653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 971/1944] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:15:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:15:20.963439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a78d5-58ae-4496-976d-44a549880811 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:20.963919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a78d5-58ae-4496-976d-44a549880811 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:20.964003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9a78d5-58ae-4496-976d-44a549880811 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:20.987826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7c9a78d5-58ae-4496-976d-44a549880811 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:20.989149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7c9a78d5-58ae-4496-976d-44a549880811 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:21.068169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 974/1945] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:15:20 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:15:21.080245 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:21.081922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:21.147748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c631554e-6cb8-490f-bcae-414663266b0a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:21.152161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:21.152772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.153051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.153323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.163929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:21.166172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6957b11b-5676-4376-aad2-d9751756e23a service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:21.177036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 972/1946] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:15:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:15:21.253266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.253552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.253811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.255186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 975/1947] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:15:21.260141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e18f9-6072-4890-a126-002fd947a4e5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.260551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e18f9-6072-4890-a126-002fd947a4e5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.260915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384e18f9-6072-4890-a126-002fd947a4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.282977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-384e18f9-6072-4890-a126-002fd947a4e5 None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:21.284677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-384e18f9-6072-4890-a126-002fd947a4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:21.292317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 973/1948] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:15:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:15:21.304099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:21.306169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:21.319175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f09324-bacc-4755-a586-7ff7196942ef None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:21.323037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:21.323628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.323912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.324187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.331174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:21.332587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7c7d7-889d-4be6-9fab-3963b9860322 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:21.341263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 976/1949] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:15:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:15:21.426380 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982615-3c9e-45d3-9125-457a28f445ff None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.427112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982615-3c9e-45d3-9125-457a28f445ff None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.427373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0982615-3c9e-45d3-9125-457a28f445ff None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.428725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 974/1950] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:15:21.434210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025d1dd-dd1a-4fd8-8c72-c8862cd14ba2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.434537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025d1dd-dd1a-4fd8-8c72-c8862cd14ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.434822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025d1dd-dd1a-4fd8-8c72-c8862cd14ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.457238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f025d1dd-dd1a-4fd8-8c72-c8862cd14ba2 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:21.458457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f025d1dd-dd1a-4fd8-8c72-c8862cd14ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:21.470329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 977/1951] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:15:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:15:21.482092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982615-3c9e-45d3-9125-457a28f445ff None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:21.494420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0982615-3c9e-45d3-9125-457a28f445ff None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:21.498699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:21.499335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:21.499712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:21.499995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:21.511837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:21.520641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82891bc4-9db9-4660-91c7-192e85769479 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:21.533003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 975/1952] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:15:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:15:30.393594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:30.394040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:30.395252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:30.421364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] MFA Rules not processed for user `66975cf6d8fa417e8df429d4c89ef910`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:30.422600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:30.482481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 978/1953] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:15:30 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:15:30.610395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ce4-aca9-4676-8b01-4f8545702388 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:30.610932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ce4-aca9-4676-8b01-4f8545702388 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:30.611062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2549ce4-aca9-4676-8b01-4f8545702388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:30.631603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f2549ce4-aca9-4676-8b01-4f8545702388 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:15:30.633375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f2549ce4-aca9-4676-8b01-4f8545702388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:15:30.640835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 976/1954] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:15:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:15:30.654183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:15:30.666404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028bd41-bcbf-43e7-b3ae-76e5e7d6c67d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:15:30.671400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:15:30.672001 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:15:30.672278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:15:30.672549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:15:30.681010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:15:30.682649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:15:30.694048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 979/1955] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:15:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:15:40.626367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=38 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:15:40.693285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=71 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:03.635265 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:03.635825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:03.635962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:03.662316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:16:03.663783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:03.673169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 977/1956] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:16:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:16:03.686138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:16:03.697938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7d5abd-5a65-4818-b4cc-2bd63dde2749 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:16:03.703287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:16:03.703894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:03.704179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:03.704455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:03.713771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:16:03.715392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:16:03.723522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 980/1957] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:16:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:16:13.651992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:13.720288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:15.985337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:16:15.995163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6845e8c-768a-4888-af9c-c7b65a5c888e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:16:15.998744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:16:15.999352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:15.999632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:15.999908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:16.001710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:16.015090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:16:16.016503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5048091b-364b-4880-ab43-bd5277eeafc4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:16:16.027630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 978/1958] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:16:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:16:17.216930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:16:17.226127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6dae636-e0ca-4226-9242-7821461bb01d service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:16:17.230424 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:16:17.231099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:17.231704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:17.232032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:17.233770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:17.247287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:16:17.248720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:16:17.259690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 981/1959] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:16:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:16:26.022383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:26.825121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:26.825407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:26.825683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:26.844711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] MFA Rules not processed for user `2335878ea1d045858b158f6b4175e896`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:16:26.845939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:26.899727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 979/1960] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:16:26 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:16:27.254641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:28.071646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:16:28.073379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:28.087374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8a9be-ba07-4593-aa22-4f62d5269c20 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:16:28.093109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:16:28.093926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:28.094209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:28.094513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:28.096235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:28.158109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:16:28.160162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:16:28.170167 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 982/1961] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:16:28 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:16:36.892310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=25 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:38.165547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=28 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:16:42.772070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:16:42.782001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f099f45c-dcb7-4c00-b402-2cadd5495661 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:16:42.785699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:16:42.786340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:16:42.786740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:16:42.787024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:16:42.788860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:16:42.798659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:16:42.800071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:16:42.808683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 980/1962] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:16:42 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:16:52.805094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:17:12.283650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:12.285899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:12.300855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8e8ce-15ef-4ad7-8649-fea8b86e8bf3 service nova] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:12.307319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:12.307929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:12.308213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:12.308491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:12.310227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:12.368489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:12.370066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c7814-5805-4a1c-b1f2-e8148d49a829 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:12.379557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 983/1963] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:15.559321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:15.573476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa9bed9-b203-4b29-8162-161564dde831 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:15.581103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:15.581736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:15.582826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:15.583105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:15.584925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:15.608860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:15.608860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:15.616443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 981/1964] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:18.906576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:18.907100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:18.907206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:18.928842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] MFA Rules not processed for user `d8cd3210dbea46d6912c93c6d28356ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:17:18.930393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:18.994470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 984/1965] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:17:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:17:19.321696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:19.330826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864839f-9637-49ed-a407-8623020ff6ad service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:19.334376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:19.335008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:19.335288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:19.335561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:19.337294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:19.346523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:19.348575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af3f02f-2a9c-440f-a652-8ff2bd5d2cee service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:19.356451 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 982/1966] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:19.818390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:19.830822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b4c011-eba0-4f1b-b511-2865d1edd06f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:19.836714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:19.837347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:19.837635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:19.837913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:19.839725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:19.851810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:19.853389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:19.863823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 985/1967] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:21.809557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:21.811290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:21.813297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:21.835609 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] MFA Rules not processed for user `c9dba6c5f7e9472faa5aa963758d78fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:17:21.836945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:21.844231 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 983/1968] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:17:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:17:29.859231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=59 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:17:31.829699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=15 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:17:34.482539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:34.497992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f6c868-a9ee-4159-b4d3-2033893dc543 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:34.505040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:34.505786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:34.506162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:34.506646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:34.509010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:34.524537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:34.526491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1382bfa6-538a-48da-ad97-3c8821f29196 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:34.539851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 986/1969] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:39.949195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:39.952857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:40.055374 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2842d790-a982-43a7-81b4-1e6c726ec597 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:40.059006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:40.059638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:40.059923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:40.060202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:40.062220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:40.075332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:40.076802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:40.087576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 984/1970] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:39 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:44.533602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:17:50.082387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=26 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:17:55.074116 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5739-44cc-4eb7-b118-bf0899421817 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:55.074646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5739-44cc-4eb7-b118-bf0899421817 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:55.074894 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6b5739-44cc-4eb7-b118-bf0899421817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:55.102277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-eb6b5739-44cc-4eb7-b118-bf0899421817 None None] MFA Rules not processed for user `66975cf6d8fa417e8df429d4c89ef910`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:17:55.102889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eb6b5739-44cc-4eb7-b118-bf0899421817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:55.116280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 987/1971] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:17:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:17:55.566849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:55.578912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af14097-012d-4554-a78e-d26e463f6935 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:55.583015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:55.583627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:55.584092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:55.584369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:55.588124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:55.604527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:55.606111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:55.615902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 985/1972] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:17:56.142422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:56.142922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:56.143018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:56.163138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] MFA Rules not processed for user `006623bab8f04f448edf0d43397192d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:17:56.164346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:56.172479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 988/1973] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:17:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:17:58.352449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:17:58.385397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833679c5-a165-45f2-9341-c0dd2f78101e service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:17:58.391868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:17:58.392489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:17:58.392772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:17:58.393293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:17:58.408439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:17:58.424399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:17:58.426082 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:17:58.435800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 986/1974] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:17:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:04.229616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:04.250186 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d573ea5c-fab7-4c1a-ba11-aff481ab9e70 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:04.256465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:04.257888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:04.258325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:04.258785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:04.260774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:04.277142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:04.281111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c57dfc9-141c-40ea-b8ed-058955ff1240 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:04.297503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 989/1975] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:05.185019 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:05.194885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e26f54-0df1-4e5f-b1ad-681708ab453e service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:05.198637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:05.199244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:05.199509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:05.199777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:05.201613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:05.267979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:05.269421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b247b2-b9de-4d95-b045-36bdd43a3203 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:05.280531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 987/1976] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:08.740626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:08.741131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:08.741223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:08.742866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 990/1977] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:18:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:18:08.761883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:08.762517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:08.763627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:08.797302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] MFA Rules not processed for user `c845b9e9ea3f43bcbab59b306d6c5da0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:08.798559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:08.807898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 988/1978] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:18:08 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:08.823222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:08.825033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:08.843549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886c72f1-eacb-4e6d-b8e8-e630d962f814 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:08.849188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:08.850135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:08.850606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:08.851392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:08.863224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:08.864695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:08.867924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:08.869653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:08.882874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06af7b1-4e3c-44e9-b4d2-cf865aa945af None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:08.888040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:08.889251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:08.889818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:08.890415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] PATH_INFO: `/v3/users/2335878ea1d045858b158f6b4175e896` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:08.892052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 991/1979] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:08.892613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2335878ea1d045858b158f6b4175e896)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:08.895447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:08.950426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2335878ea1d045858b158f6b4175e896'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:08.959965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2335878ea1d045858b158f6b4175e896'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:08.960972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2335878ea1d045858b158f6b4175e896'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:08.972190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2335878ea1d045858b158f6b4175e896'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:08.972989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eab56165-07a6-4634-9c2e-a23604a1f59c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2335878ea1d045858b158f6b4175e896'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.005468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 989/1980] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:08 2022] DELETE /identity/v3/users/2335878ea1d045858b158f6b4175e896 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:09.183601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.203754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7007d991-4e5d-4a81-b76e-af226d02f56e service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.271477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.272377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.272655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.272926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] PATH_INFO: `/v3/projects/d7ac3d840c9f41ea842b8f4935770f8b` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.274040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ac3d840c9f41ea842b8f4935770f8b)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.276005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.321108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ac3d840c9f41ea842b8f4935770f8b', 'request_id': 'req-a8675d7b-2818-435c-8602-d52ce11ddd40'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.327864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ac3d840c9f41ea842b8f4935770f8b', 'request_id': 'req-a8675d7b-2818-435c-8602-d52ce11ddd40'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.333275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ac3d840c9f41ea842b8f4935770f8b', 'request_id': 'req-a8675d7b-2818-435c-8602-d52ce11ddd40'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.345701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] The token cache is being invalidate because project d7ac3d840c9f41ea842b8f4935770f8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:18:09.346011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.346666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a8675d7b-2818-435c-8602-d52ce11ddd40 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:09.348137 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 992/1981] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:18:09 2022] DELETE /identity/v3/projects/d7ac3d840c9f41ea842b8f4935770f8b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:09.353100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.353331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.353607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.374655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:09.375851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:09.405460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:09.437039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 990/1982] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:09.444546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.444836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.445113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.465586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:09.466875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:09.475080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 993/1983] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:09.479425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.489575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ac8cb4-0249-4277-a8d5-8166c356855a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.494281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.494927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.495204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.495593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.496240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:18:09.496543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.499121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.510632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 991/1984] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:18:09.516102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.525840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600b0da-24c4-4db2-b268-d3cbb06a7be0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.530536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.531139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.531421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.531691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.532317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.534199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.573280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 994/1985] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:18:09 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:09.574960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.589590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d351ea5c-c908-4405-acb1-fac4a918b68e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.649229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.649852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.650135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.650409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.651061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.652950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.674323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 992/1986] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:18:09 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:09.678765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.689885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752df4e6-fc1d-4332-a91b-3fcaeb14f5e8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.694519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.695124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.695405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.695680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.696403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.697930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.708282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 995/1987] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:09.712368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.722487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f7c23e-a10e-4980-b55b-85a790fe9d75 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.727126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.727744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.728036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.728309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.728943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.730578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a76ae1-bb64-494e-8a16-2c2408c818cc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.743968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 993/1988] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:09.749341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.760860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4cdbfa-73b2-4306-86cc-5a30857221cd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:09.766458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:09.767241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.767684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.767978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] PATH_INFO: `/v3/projects/a29518c15f6e4b75a63ae696827c6db2/users/32a2abd2e7be48f9b42ef80fa73eb2bf/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.777099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:09.786382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a29518c15f6e4b75a63ae696827c6db2, user_id=32a2abd2e7be48f9b42ef80fa73eb2bf, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:09.790005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:09.813126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 996/1989] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:18:09 2022] PUT /identity/v3/projects/a29518c15f6e4b75a63ae696827c6db2/users/32a2abd2e7be48f9b42ef80fa73eb2bf/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:18:09.819821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:09.820346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:09.820555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:09.850867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] MFA Rules not processed for user `32a2abd2e7be48f9b42ef80fa73eb2bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:09.852567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:09.929811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 994/1990] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:09.942931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:09.945120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:10.033515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375f3a4d-bbd1-4bab-8d8b-92ae5e0614d4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:10.040904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:10.041657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:10.042841 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:10.043128 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:10.056118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:10.057814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9573c1-0549-4863-9f44-966cb05b69d8 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:10.069304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 997/1991] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:19.594415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:19.627484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:19.665493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296b558-69f1-48e8-a780-14d11ac9cea0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:19.670757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:19.671395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:19.671698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:19.671938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] PATH_INFO: `/v3/users/32a2abd2e7be48f9b42ef80fa73eb2bf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:19.673064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] RBAC: Authorizing `identity:delete_user(user_id=32a2abd2e7be48f9b42ef80fa73eb2bf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:19.675179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:19.728074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32a2abd2e7be48f9b42ef80fa73eb2bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:19.736365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32a2abd2e7be48f9b42ef80fa73eb2bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:19.737089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32a2abd2e7be48f9b42ef80fa73eb2bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:19.752799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32a2abd2e7be48f9b42ef80fa73eb2bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:19.753450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32a2abd2e7be48f9b42ef80fa73eb2bf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:19.786020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 995/1992] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:19 2022] DELETE /identity/v3/users/32a2abd2e7be48f9b42ef80fa73eb2bf => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:19.790876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:19.791381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:19.791498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:19.812572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:19.813790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:19.883181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 998/1993] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:19.894775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:19.896664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:19.967103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36bba92-68a9-4011-a033-c17847a88eac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:19.970955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:19.971573 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:19.971859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:19.972132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:19.987192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:19.988662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a261438c-d513-4a08-a0c6-ac3446d09c23 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.000404 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 996/1994] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:20.360654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.362471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:20.375727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f68d51-04c7-4c1d-b786-a5a6926f6daf None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.382279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.383040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.383391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.383757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] PATH_INFO: `/v3/projects/a29518c15f6e4b75a63ae696827c6db2` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.385087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a29518c15f6e4b75a63ae696827c6db2)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.387165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.439246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a29518c15f6e4b75a63ae696827c6db2', 'request_id': 'req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:20.446123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a29518c15f6e4b75a63ae696827c6db2', 'request_id': 'req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:20.452971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a29518c15f6e4b75a63ae696827c6db2', 'request_id': 'req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:20.456968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:20.469755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] The token cache is being invalidate because project a29518c15f6e4b75a63ae696827c6db2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:18:20.470044 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:20.470695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6df9bdd8-638c-4d30-a00b-296b66d0ed6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:20.472603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 999/1995] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:18:20 2022] DELETE /identity/v3/projects/a29518c15f6e4b75a63ae696827c6db2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:20.477319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.477614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.477895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.502237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:20.503539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:20.551182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:20.568177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 997/1996] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:20.576150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.576437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.576711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.597515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:20.598730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:20.608159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1000/1997] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:20.612339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.622166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6505a93e-d802-4913-9f44-6af2bc96dea5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.627360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.627974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.628256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.628530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.629207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:18:20.629507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.631215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.643763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 998/1998] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:18:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:18:20.648201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.657959 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba53f0e-4f18-4ac9-b3b5-c201297a8495 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.662878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.663477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.663756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.664026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.664663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.666571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.700936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1001/1999] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:18:20 2022] POST /identity/v3/projects => generated 367 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:20.704603 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.714687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3d254-853a-4f2d-8263-acea7d955849 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.774156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.774901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.775251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.775655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.776556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.779464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.806751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 999/2000] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:18:20 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:18:20.807983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.819277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d14573-3727-476a-9797-94e880784d56 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.824612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.825219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.825653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.826663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.827213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.828986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.839824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1002/2001] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:20.844858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.855098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a88fede-f03c-48fc-8aac-31c6b4778163 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.860785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.861399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.861683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.861965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.862637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.864253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe4ea5-5f7f-4d9b-905d-1bf0ed995aa4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.875377 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1000/2002] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:20.879605 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:20.889686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d962d3b-115d-4ef4-b595-fde2ceda0e72 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:20.894577 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:20.895249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.895601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.896216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] PATH_INFO: `/v3/projects/665fb029e2554b58972d49198aa485a4/users/89dc98d72c434f758443c7225c226137/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.912682 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] RBAC: Authorizing `identity:create_grant(project_id=665fb029e2554b58972d49198aa485a4, user_id=89dc98d72c434f758443c7225c226137, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:20.915453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:20.930333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1003/2003] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:18:20 2022] PUT /identity/v3/projects/665fb029e2554b58972d49198aa485a4/users/89dc98d72c434f758443c7225c226137/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:20.940867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:20.941190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:20.941472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:20.975389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] MFA Rules not processed for user `89dc98d72c434f758443c7225c226137`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:20.976707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:21.044383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1001/2004] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:20 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:18:21.055543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:21.057206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:21.146750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef691fb0-b73f-48b5-98be-ec56e3b0e608 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:21.150336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:21.151026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:21.151313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:21.151588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:21.161690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:21.163190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c144fd8-9c1f-4820-a8b8-6a321502675e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:21.173953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1004/2005] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:29.935929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:29.984372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:30.000875 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45cfc8ad-922d-4e47-971a-61c88c40e120 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:30.005293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:30.005920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:30.006198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:30.006495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] PATH_INFO: `/v3/users/89dc98d72c434f758443c7225c226137` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:30.007662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] RBAC: Authorizing `identity:delete_user(user_id=89dc98d72c434f758443c7225c226137)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:30.009556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:30.057419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89dc98d72c434f758443c7225c226137'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.066258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89dc98d72c434f758443c7225c226137'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.066990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89dc98d72c434f758443c7225c226137'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.079158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89dc98d72c434f758443c7225c226137'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.079868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89dc98d72c434f758443c7225c226137'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.116260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1002/2006] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:29 2022] DELETE /identity/v3/users/89dc98d72c434f758443c7225c226137 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:30.120624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:30.121143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:30.121303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:30.140346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:30.141554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:30.203885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1005/2007] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:30.213364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:30.215630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:30.272633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876d8c5a-9692-431b-87c8-66c5a132fd4a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:30.276255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:30.276855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:30.277663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:30.277938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:30.286822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:30.288230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37677831-bf87-4e6b-9dee-f82575060ba4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:30.298088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1003/2008] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:30.819158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:30.820964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:30.834569 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036e03d7-1850-4d1a-9538-cf85dc7d73b1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:30.839150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:30.839883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:30.840165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:30.840445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] PATH_INFO: `/v3/projects/665fb029e2554b58972d49198aa485a4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:30.841578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] RBAC: Authorizing `identity:delete_project(project_id=665fb029e2554b58972d49198aa485a4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:30.843604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:30.878878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:30.893860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '665fb029e2554b58972d49198aa485a4', 'request_id': 'req-047546a1-73c4-405a-983e-5ae986ce8357'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.900452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '665fb029e2554b58972d49198aa485a4', 'request_id': 'req-047546a1-73c4-405a-983e-5ae986ce8357'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.906302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '665fb029e2554b58972d49198aa485a4', 'request_id': 'req-047546a1-73c4-405a-983e-5ae986ce8357'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.918091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] The token cache is being invalidate because project 665fb029e2554b58972d49198aa485a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:18:30.918490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.919117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-047546a1-73c4-405a-983e-5ae986ce8357 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:30.920599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1006/2009] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:18:30 2022] DELETE /identity/v3/projects/665fb029e2554b58972d49198aa485a4 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:30.925736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:30.926025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:30.926359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:30.946075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:30.947237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:31.005208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1004/2010] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:31.013705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.013988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.014265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.033826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:31.035042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:31.042685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1007/2011] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:31.046732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.056243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac55182a-e091-4a1c-a4c0-71ca8bb84690 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.061592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.062190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.062503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.062781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.063415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:18:31.063722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.065306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.067628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:31.078991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1005/2012] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:18:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:18:31.083106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.092836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a4cda8-8913-482d-8518-40ac99070b91 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.097486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.098078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.098355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.098658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.099277 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.101188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.128715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1008/2013] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:18:31 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:31.132891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.143475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d9db70-6b71-475b-a473-f0cef540c807 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.200503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.201060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.201338 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.201686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.202282 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.204205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.228424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1006/2014] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:18:31 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:31.229627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.240266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41963b4-8b3d-4d44-9cd2-ffb0ba634488 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.244845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.245443 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.245831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.246650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.247294 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.248874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.258964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1009/2015] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:31.263218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.273575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df733180-8cfa-4bf2-bd05-b73fb7aa1fd5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.278310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.278975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.279255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.279532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.280173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.281792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a49b62-5619-44ad-9340-4ea3b9afabb6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.296908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1007/2016] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:18:31.298111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.307984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4164c53-d28b-4ad4-8aa5-f38bb634eeaa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.313230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.313876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.314157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.314431 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] PATH_INFO: `/v3/projects/cf9e0b8c315d44369bcbb62d8ba14308/users/f44bce2010c249f2bd67108d86366258/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.329964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9e0b8c315d44369bcbb62d8ba14308, user_id=f44bce2010c249f2bd67108d86366258, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.332660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.346157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1010/2017] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:18:31 2022] PUT /identity/v3/projects/cf9e0b8c315d44369bcbb62d8ba14308/users/f44bce2010c249f2bd67108d86366258/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:31.357070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.357494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.357839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.386954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] MFA Rules not processed for user `f44bce2010c249f2bd67108d86366258`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:31.388283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:31.452607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1008/2018] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:31.464000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.466832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:31.537896 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25d7853-f56e-4f07-bf2c-453856d3e28a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.543753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.544354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.544637 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.544915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.552375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.553947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c436fd5-ece5-461b-ba19-767dedf50850 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.563235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1011/2019] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:31 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:31.917006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:31.981530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980504c-efea-4f63-8f72-8b8954f27f9c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:31.986434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:31.987123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:31.987411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:31.987714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] PATH_INFO: `/v3/users/f44bce2010c249f2bd67108d86366258` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:31.988970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f44bce2010c249f2bd67108d86366258)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:31.990923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:31.995767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:32.041190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f44bce2010c249f2bd67108d86366258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.050590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f44bce2010c249f2bd67108d86366258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.051381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f44bce2010c249f2bd67108d86366258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.061816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f44bce2010c249f2bd67108d86366258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.062619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f44bce2010c249f2bd67108d86366258'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.081035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1009/2020] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:31 2022] DELETE /identity/v3/users/f44bce2010c249f2bd67108d86366258 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:32.085594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.085969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.086333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.106121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:32.107515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:32.144518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:32.172358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1012/2021] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:32.182114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:32.184793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:32.243138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca80a8fd-6a4a-4ef0-8c51-fe76c15b5273 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:32.247053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:32.247665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.247979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.248257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.257826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:32.259480 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12729ff8-7f09-4276-802e-3fcc9ab79366 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:32.270381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1010/2022] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:32.566803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:32.568304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:32.581742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8c7bb-aab3-4340-ac49-cff93ed0cfcb None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:32.591656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:32.592461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.592872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.593155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] PATH_INFO: `/v3/projects/cf9e0b8c315d44369bcbb62d8ba14308` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.595032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9e0b8c315d44369bcbb62d8ba14308)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:32.597111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:32.644537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9e0b8c315d44369bcbb62d8ba14308', 'request_id': 'req-a75fe0e5-f529-455f-99e4-50268df9ed9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.650893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9e0b8c315d44369bcbb62d8ba14308', 'request_id': 'req-a75fe0e5-f529-455f-99e4-50268df9ed9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.657016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9e0b8c315d44369bcbb62d8ba14308', 'request_id': 'req-a75fe0e5-f529-455f-99e4-50268df9ed9b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.669574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] The token cache is being invalidate because project cf9e0b8c315d44369bcbb62d8ba14308 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:18:32.669859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.670507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a75fe0e5-f529-455f-99e4-50268df9ed9b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:32.672080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1013/2023] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:18:32 2022] DELETE /identity/v3/projects/cf9e0b8c315d44369bcbb62d8ba14308 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:32.678221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.678565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.678846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.709032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:32.710225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:32.769256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1011/2024] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:32.778295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.778703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.778985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.798872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:32.800075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:32.807975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1014/2025] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:32.812833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:32.822274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02599785-2f9e-4823-acfb-b989c61cca88 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:32.827254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:32.827851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.828134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.828408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.829041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:18:32.829346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:32.831031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:32.842502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1012/2026] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:18:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:18:32.846924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:32.856941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcef7e6-f9ef-4f0a-91e9-1d2b11db4f1b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:32.861587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:32.862312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.862627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.862899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.863554 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:32.865590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:32.892986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1015/2027] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:18:32 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:32.897482 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:32.907734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b497729-568c-435f-82ef-76fff56797b4 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:32.916493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:32.965905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:32.966523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:32.966803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:32.967073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:32.967687 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:32.969574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:32.992014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1013/2028] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:18:32 2022] POST /identity/v3/users => generated 281 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:18:32.996563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.007094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38529a0-f7ce-431f-ae89-b13fae394b2f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.011764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.012354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.012634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.012917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.013564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.015199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.025284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1016/2029] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:33.029201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.038907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30e6750-a773-4dc0-95e0-30aed258e858 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.044189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.044783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.045064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.045339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.045967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.047592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4770e5-f0a2-484a-a267-9f61bb9757f6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.057942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1014/2030] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:33.062471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.072365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9300e1fe-df0a-4226-95d7-076119538241 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.077338 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.077986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.078270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.078590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] PATH_INFO: `/v3/projects/e9cc6952074f40e0b368831eeb98a574/users/4632dc2acc3e4d409ee31368ac482b9e/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.093344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9cc6952074f40e0b368831eeb98a574, user_id=4632dc2acc3e4d409ee31368ac482b9e, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.096031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.109535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1017/2031] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:18:33 2022] PUT /identity/v3/projects/e9cc6952074f40e0b368831eeb98a574/users/4632dc2acc3e4d409ee31368ac482b9e/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:18:33.119682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.119974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.120297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.147886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] MFA Rules not processed for user `4632dc2acc3e4d409ee31368ac482b9e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:33.149153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:33.223029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1015/2032] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:33.261593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.263984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:33.323319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fe3358-3d1f-489e-b18e-b2993fcc9dac admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.326906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.327510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.327791 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.328064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.336119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.337546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca31ccf-44a3-465d-a7d6-acbfbe39171b service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.346686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1018/2033] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:33.363215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.425647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b605bc27-3a7b-487d-93d9-127a564a4561 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.430171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.430818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.431100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.431373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] PATH_INFO: `/v3/users/4632dc2acc3e4d409ee31368ac482b9e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.432497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4632dc2acc3e4d409ee31368ac482b9e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.434398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.457522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:33.491132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4632dc2acc3e4d409ee31368ac482b9e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:33.498664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4632dc2acc3e4d409ee31368ac482b9e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:33.499384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4632dc2acc3e4d409ee31368ac482b9e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:33.506495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4632dc2acc3e4d409ee31368ac482b9e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:33.507125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4632dc2acc3e4d409ee31368ac482b9e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:33.524515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1016/2034] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:33 2022] DELETE /identity/v3/users/4632dc2acc3e4d409ee31368ac482b9e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:33.529226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.529519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.529800 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.536885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:33.551154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:33.552348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:33.612905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1019/2035] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:33.623820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.625212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:33.689235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c617847-5029-41c1-a5f2-cb3827dd69f1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:33.693251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:33.693860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:33.694136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:33.694420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:33.705678 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:33.707215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799131-a538-4073-b27f-bd578a3f16d3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:33.717986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1017/2036] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:33.987286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:33.988687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:34.001855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea1d81d-7b85-4178-a6b2-c17c98948055 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.006486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.007119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.007395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.007665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] PATH_INFO: `/v3/projects/e9cc6952074f40e0b368831eeb98a574` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.008825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9cc6952074f40e0b368831eeb98a574)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.010747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.056606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9cc6952074f40e0b368831eeb98a574', 'request_id': 'req-864a628c-51ce-4692-9c59-bb84ebbe5b28'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:34.063403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9cc6952074f40e0b368831eeb98a574', 'request_id': 'req-864a628c-51ce-4692-9c59-bb84ebbe5b28'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:34.069127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9cc6952074f40e0b368831eeb98a574', 'request_id': 'req-864a628c-51ce-4692-9c59-bb84ebbe5b28'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:34.081806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] The token cache is being invalidate because project e9cc6952074f40e0b368831eeb98a574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:18:34.082092 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:34.082773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-864a628c-51ce-4692-9c59-bb84ebbe5b28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:34.084273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1020/2037] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:18:33 2022] DELETE /identity/v3/projects/e9cc6952074f40e0b368831eeb98a574 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:34.089274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.089758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.089854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.109393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:34.110612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:34.167219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1018/2038] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:34.175576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.175865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.176143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.196476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:34.197680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:34.205799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1021/2039] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:18:34.209946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.219625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd860f91-9d3f-4f12-95f6-34284d45eb41 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.224330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.224976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.225261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.225536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.226178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:18:34.226511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.228111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.240025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1019/2040] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:18:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:18:34.244404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.254360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d640229-2546-415c-9ec0-7e8d16137b1f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.259069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.259671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.259951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.260281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.260952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.263721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.293376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1022/2041] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:18:34 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:34.298884 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.308821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63345187-89d9-40ee-8e18-26f6e774eafe admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.338717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:34.371281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.371949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.372862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.373140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.373778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.375762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.397055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1020/2042] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:18:34 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:18:34.401432 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.411957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67203df0-0de8-4016-b067-2bf81097f230 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.416614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.417221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.417500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.417773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.418407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.420692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.430457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1023/2043] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:34.434348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.445148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1194c394-4688-4854-97d6-7e68a61f7338 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.449846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.451162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.451375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.451679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.452334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.453926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f001b475-11e8-4bc6-be79-72a6404f4d1c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.466974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1021/2044] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:18:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:18:34.471170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.481120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d88fc4-eb77-4744-88e6-21a66f53d327 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.486192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.486875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.487149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.487419 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] PATH_INFO: `/v3/projects/b16d0e1c22874464abf5b066c4e94817/users/ec32985f6f0c42cea7148ac63adaeba3/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.501998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16d0e1c22874464abf5b066c4e94817, user_id=ec32985f6f0c42cea7148ac63adaeba3, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.504674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.518341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1024/2045] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:18:34 2022] PUT /identity/v3/projects/b16d0e1c22874464abf5b066c4e94817/users/ec32985f6f0c42cea7148ac63adaeba3/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:18:34.530398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.530747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.531047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.558162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] MFA Rules not processed for user `ec32985f6f0c42cea7148ac63adaeba3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:34.559912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:34.623164 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1022/2046] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:18:34.634230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:34.635936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:34.663972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:34.696946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046605f-8b46-436a-b3fb-7419f550ea97 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:34.700885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:34.701497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:34.701776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:34.702049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:34.710279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:34.711738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107334b5-810f-4bb8-ae8d-188aff0a1217 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:34.721119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1025/2047] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:18:44.620195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=52,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:44.717320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=16 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:18:59.560102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:59.621833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e07fc6-355f-4b9c-ac39-6b224f93c7de None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:59.626360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:59.627080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:59.627364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:59.627666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] PATH_INFO: `/v3/users/ec32985f6f0c42cea7148ac63adaeba3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:59.628785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec32985f6f0c42cea7148ac63adaeba3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:59.630734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:59.676956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec32985f6f0c42cea7148ac63adaeba3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:59.684333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec32985f6f0c42cea7148ac63adaeba3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:59.685137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec32985f6f0c42cea7148ac63adaeba3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:59.693074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec32985f6f0c42cea7148ac63adaeba3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:59.693858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec32985f6f0c42cea7148ac63adaeba3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:18:59.711808 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1023/2048] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:18:59 2022] DELETE /identity/v3/users/ec32985f6f0c42cea7148ac63adaeba3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:18:59.716888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:59.717361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:59.717467 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:59.736195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:18:59.737376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:59.794253 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1026/2049] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:18:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:18:59.803886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:18:59.805506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:18:59.861657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4c9fa0-d1ad-49b1-b32d-819037de1bb5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:18:59.865227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:18:59.865831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:18:59.866115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:18:59.866384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:18:59.875392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:18:59.876804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d3841a-d2fd-457f-ad23-3374071ceb6e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:18:59.885922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1024/2050] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:19:00.144840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.146270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:00.162718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00f86bb-19e6-4674-b877-b335d7c8ae91 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.168638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.169282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.169662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.169820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] PATH_INFO: `/v3/projects/b16d0e1c22874464abf5b066c4e94817` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.171204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16d0e1c22874464abf5b066c4e94817)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.173088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.219942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16d0e1c22874464abf5b066c4e94817', 'request_id': 'req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:00.226226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16d0e1c22874464abf5b066c4e94817', 'request_id': 'req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:00.231980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16d0e1c22874464abf5b066c4e94817', 'request_id': 'req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:00.244310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] The token cache is being invalidate because project b16d0e1c22874464abf5b066c4e94817 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:19:00.244690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:00.245309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9e2efc20-8ce7-4373-85d8-b7a2a99b41ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:00.247363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1027/2051] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:19:00 2022] DELETE /identity/v3/projects/b16d0e1c22874464abf5b066c4e94817 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:00.252388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.252701 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.252979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.273490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:00.274718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:00.291645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:00.334885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1025/2052] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:00.343381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.343665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.343937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.364073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:00.365264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:00.373135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1028/2053] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:00.377327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.387774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a605bf-6991-465e-9cc0-660f5568535a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.392353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.393199 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.393512 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.393788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.394431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:19:00.394804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.396405 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.408625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1026/2054] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:19:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:19:00.412819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.422202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a41858-dee4-4932-a49a-30418ea62379 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.426873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.427463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.427740 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.428010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.428639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.430583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.459126 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1029/2055] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:19:00 2022] POST /identity/v3/projects => generated 397 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:19:00.463466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.473915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52201da-3b80-46a3-ad59-95bb458a44e1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.529119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.529647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.529924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.530207 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.530863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.532758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.553260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1027/2056] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:19:00 2022] POST /identity/v3/users => generated 299 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:19:00.557446 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.567987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f33371-7240-4406-9307-ce082696f6fa admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.572565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.573146 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.573422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.574549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.575206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.576805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.586491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1030/2057] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:00.591012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.601272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93002e33-2432-40fc-8c8b-36b18c5340c2 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.606552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.607145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.607430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.607707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.608341 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.609935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805eeb11-c3f4-468e-8c25-f9c532b2c656 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.620343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1028/2058] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:00.624905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.634711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8155e1-4c18-4273-a3de-ca60ecb0c5a4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.639549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.640169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.640447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.640720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] PATH_INFO: `/v3/projects/c214609a4d5d405eaa70ed8fac957120/users/43d96bf13ccd4ed998eb91f2e907adf5/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.655142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c214609a4d5d405eaa70ed8fac957120, user_id=43d96bf13ccd4ed998eb91f2e907adf5, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.657762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.670634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1031/2059] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:19:00 2022] PUT /identity/v3/projects/c214609a4d5d405eaa70ed8fac957120/users/43d96bf13ccd4ed998eb91f2e907adf5/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:00.682081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.682373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.682694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.710948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] MFA Rules not processed for user `43d96bf13ccd4ed998eb91f2e907adf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:00.712182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:00.775533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1029/2060] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:00 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:00.787192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:00.788904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:00.810710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:00.846582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c73364-e186-49ab-9177-1faf3a0d91e6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:00.850224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:00.850945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:00.851250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:00.851521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:00.860247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:00.861663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ace10-333d-4192-8f16-6d742a2c3f2e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:00.870349 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1032/2061] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:19:00 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:19:08.945845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:08.965446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:09.016607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f7ce34-45e0-4aea-8d7f-a0da9fb17cad None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.021393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.022015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.022299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.022610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] PATH_INFO: `/v3/users/43d96bf13ccd4ed998eb91f2e907adf5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.023724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d96bf13ccd4ed998eb91f2e907adf5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.025633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.071266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d96bf13ccd4ed998eb91f2e907adf5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.078473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d96bf13ccd4ed998eb91f2e907adf5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.079141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d96bf13ccd4ed998eb91f2e907adf5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.086553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d96bf13ccd4ed998eb91f2e907adf5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.087184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d96bf13ccd4ed998eb91f2e907adf5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.104252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1030/2062] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:19:08 2022] DELETE /identity/v3/users/43d96bf13ccd4ed998eb91f2e907adf5 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:09.108659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.109191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.109191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.128378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:09.129566 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:09.188673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1033/2063] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:09.198267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.199714 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:09.258183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5e5edd-c826-463c-beb6-3e9753ef20ca admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.261805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.262386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.263312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.263803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.272911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.274336 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a1bbc-eba3-4722-81e5-eb22aff398f3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.283647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1031/2064] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:19:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:19:09.547032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.548758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:09.561464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10939b49-194c-45ff-a540-3b70b0da7230 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.566049 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.566712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.567000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.567278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] PATH_INFO: `/v3/projects/c214609a4d5d405eaa70ed8fac957120` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.568910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] RBAC: Authorizing `identity:delete_project(project_id=c214609a4d5d405eaa70ed8fac957120)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.570528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.616956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c214609a4d5d405eaa70ed8fac957120', 'request_id': 'req-6f9013c5-d1b2-4b00-a809-1bb27a169dee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.623809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c214609a4d5d405eaa70ed8fac957120', 'request_id': 'req-6f9013c5-d1b2-4b00-a809-1bb27a169dee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.629313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c214609a4d5d405eaa70ed8fac957120', 'request_id': 'req-6f9013c5-d1b2-4b00-a809-1bb27a169dee'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.641823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] The token cache is being invalidate because project c214609a4d5d405eaa70ed8fac957120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:19:09.642134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.643096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6f9013c5-d1b2-4b00-a809-1bb27a169dee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:19:09.644613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1034/2065] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:19:09 2022] DELETE /identity/v3/projects/c214609a4d5d405eaa70ed8fac957120 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:09.649595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.649881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.650154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.670472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:09.671681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:09.707075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:09.734398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1032/2066] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:09.742152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.742445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.742746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.762402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:09.763680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:09.771862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1035/2067] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:09.775948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.786431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c300591f-0c08-4f81-985a-41a05bdf3471 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.790992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.791576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.791855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.792127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.792758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:19:09.793061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.795878 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.807099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1033/2068] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:19:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:19:09.811259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.820172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a8d2a-35f3-4cef-b10d-d2325303efcc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.824762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.825539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.826163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.826433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.827089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.828956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.857639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1036/2069] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:19:09 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:19:09.861738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.871684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847b8895-3294-4b92-8c21-bf00cb20cfb8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.928501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.929102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.929386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.929667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.930287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.932235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.954352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1034/2070] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:19:09 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:19:09.958490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:09.969084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed5606d-2a79-482b-a1d6-57e6ca572a4c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:09.973556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:09.974138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:09.974414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:09.974716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:09.975340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:09.976934 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:09.987375 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1037/2071] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:09.991150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:10.001034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d693d7b-de5c-4bdd-9a1a-e69d7d79e29a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:10.005645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:10.006236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:10.006558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:10.006845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:10.007485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:10.009077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98993a6-ab21-4c20-8dc3-a709f201d784 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:10.019727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1035/2072] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:10.023858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:10.034057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66fd2-9f2b-4477-9242-74d95f36e088 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:10.038552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:10.039180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:10.039459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:10.039736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] PATH_INFO: `/v3/projects/7d85fefc9f0b47f8bad45fb6dd0810f9/users/7a3de8194ccd4eec9c19a6b250944457/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:10.042314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:10.057437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d85fefc9f0b47f8bad45fb6dd0810f9, user_id=7a3de8194ccd4eec9c19a6b250944457, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:10.060196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:10.072762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1038/2073] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:19:10 2022] PUT /identity/v3/projects/7d85fefc9f0b47f8bad45fb6dd0810f9/users/7a3de8194ccd4eec9c19a6b250944457/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:19:10.078319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967805f1-8630-4085-897d-35a1e7878136 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:10.078651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967805f1-8630-4085-897d-35a1e7878136 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:10.078928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967805f1-8630-4085-897d-35a1e7878136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:10.098091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-967805f1-8630-4085-897d-35a1e7878136 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:10.099327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-967805f1-8630-4085-897d-35a1e7878136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:10.157347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1036/2074] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:19:10.167260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:10.169016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:10.225995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2e3ef9-c190-4695-a27e-f7fe23d20465 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:10.229535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:10.230135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:10.230417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:10.230743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:10.239754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:10.241111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:10.250369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1039/2075] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:19:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:19:14.114991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.115536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.115536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.144247 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] MFA Rules not processed for user `7a3de8194ccd4eec9c19a6b250944457`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:14.145299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:14.182346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:14.218027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1037/2076] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:14.220353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.230073 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54235355-a1b7-4713-9640-aefd7e105428 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.234856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.235451 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.235731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.236009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.236652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.238573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.266517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1040/2077] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:19:14 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:19:14.270703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.281561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aeb194-6ad9-488c-b479-e409f01c5585 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.343287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.343940 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.344278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.344543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.345180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.347130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.368949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1038/2078] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:19:14 2022] POST /identity/v3/users => generated 299 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:19:14.373489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.384046 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cce30db-dab4-4ea6-b7e8-16df2995e448 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.389350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.389999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.390245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.390552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.391633 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.394150 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.406244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1041/2079] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:14.410431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.421721 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e5685-a244-4c27-a6f5-f7889cab77b8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.426670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.427354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.427635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.427912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] PATH_INFO: `/v3/projects/29a8b1e914054f14a418ed039f8a1a4f/users/e43c3e4d87e844b3aa9b3ce31155b802/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.448937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a8b1e914054f14a418ed039f8a1a4f, user_id=e43c3e4d87e844b3aa9b3ce31155b802, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.452271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.477465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1039/2080] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:19:14 2022] PUT /identity/v3/projects/29a8b1e914054f14a418ed039f8a1a4f/users/e43c3e4d87e844b3aa9b3ce31155b802/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:14.482712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.499095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158aa29d-40a1-4b70-ac94-3840fa06b4b7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.563266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.563889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.564172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.564465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.565112 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.566842 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871d8814-453c-4695-822f-75e118c9b725 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.578695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1042/2081] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:19:14 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:19:14.582789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:14.593948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2236707-11f6-4f80-8613-91730251c5f6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:14.600938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:14.601617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:14.601905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:14.602185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] PATH_INFO: `/v3/projects/29a8b1e914054f14a418ed039f8a1a4f/users/e43c3e4d87e844b3aa9b3ce31155b802/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:14.604183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a8b1e914054f14a418ed039f8a1a4f, user_id=e43c3e4d87e844b3aa9b3ce31155b802, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:14.607652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:14.619682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1040/2082] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:19:14 2022] PUT /identity/v3/projects/29a8b1e914054f14a418ed039f8a1a4f/users/e43c3e4d87e844b3aa9b3ce31155b802/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:19:19.059184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2cdc-baed-4a06-9bf7-c0a9130d7448 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:19.059725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2cdc-baed-4a06-9bf7-c0a9130d7448 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:19.059725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9f2cdc-baed-4a06-9bf7-c0a9130d7448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:19.108558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3e9f2cdc-baed-4a06-9bf7-c0a9130d7448 None None] MFA Rules not processed for user `e43c3e4d87e844b3aa9b3ce31155b802`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:19.109741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3e9f2cdc-baed-4a06-9bf7-c0a9130d7448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:19.140708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:19.207701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1043/2083] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:19 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:19.221571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:19.223589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:19.333063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4ae0c9-c27c-40a8-9b42-332f0b7ef3a1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:19.336859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:19.337489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:19.337774 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:19.338047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:19.364949 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:19.404281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:19.405734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:19.415249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1041/2084] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:19:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:19:19.574757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c049c53-906e-4a89-896b-d45646417ec1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:19.575319 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c049c53-906e-4a89-896b-d45646417ec1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:19.575606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c049c53-906e-4a89-896b-d45646417ec1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:19.612063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0c049c53-906e-4a89-896b-d45646417ec1 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:19.613353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0c049c53-906e-4a89-896b-d45646417ec1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:19.728677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1044/2085] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:19:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:19.785384 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:19.786890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:19.863009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d556bb-64a0-4ee2-b9ee-e01d2a5d4684 service glance] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:19.868361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:19.868985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:19.869278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:19.869555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:19.878523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:19.880127 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5b81e2-e96c-43b7-9f1d-1ed84538ad9d service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:19.890109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1042/2086] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:19:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:19:28.691444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:28.691920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:28.692004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:28.711306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] MFA Rules not processed for user `7a3de8194ccd4eec9c19a6b250944457`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:28.712524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:28.720163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1045/2087] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:19:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:29.885358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=49 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:31.076594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:31.077184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:31.077184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:31.096728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:19:31.097820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:19:31.104957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1043/2088] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:19:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:19:31.120004 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:19:31.131869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48197c4f-8919-4144-93df-9f7aeb133acc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:19:31.137498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:19:31.138283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:19:31.138771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:19:31.139045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:19:31.150730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:19:31.153050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:19:31.161399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1046/2089] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:19:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:19:41.091134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:19:41.159317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=68 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:03.849996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:03.851408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:03.927851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cfe8bc-78c2-4bef-ba2f-a974e59b74ba None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:03.931387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:03.931982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:03.932259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:03.932531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:03.934224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:03.991067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:03.992419 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07698418-584a-4b75-b00e-6483c5d83a75 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:04.001724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1044/2090] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:03 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:11.609398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:11.675056 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84966de1-34f9-48c1-a450-6e4d14443415 service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:11.678938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:11.681822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:11.681822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:11.681822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:11.696539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:11.698034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:11.708193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1047/2091] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:13.312631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04323f2e-b735-4c2b-91a5-06ae339e6ff1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:13.313211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04323f2e-b735-4c2b-91a5-06ae339e6ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:13.313211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04323f2e-b735-4c2b-91a5-06ae339e6ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:13.333564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-04323f2e-b735-4c2b-91a5-06ae339e6ff1 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:13.334694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-04323f2e-b735-4c2b-91a5-06ae339e6ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:13.342649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1045/2092] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:20:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:20:13.355308 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:13.367041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac3af09-f1b4-4637-ae83-253a01ba6a36 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:13.372140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:13.372737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:13.373070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:13.373537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:13.381878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:13.383578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:13.392629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1048/2093] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:20:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:20:18.309797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:18.310776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:18.311242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:18.330671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] MFA Rules not processed for user `d8cd3210dbea46d6912c93c6d28356ae`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:18.332056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:18.339065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1046/2094] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:20:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:20:19.855574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:19.857303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:19.953257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6015f672-933a-4799-b0a2-0ffe2e5a834a service swift] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:19.958160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:19.958784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:19.959115 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:19.959320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:19.961512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:19.973420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:19.975169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:19.986173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1049/2095] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:23.676226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:23.677743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:23.765729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498a9d12-6d49-42a1-a5af-be649f07df7a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:23.775533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:23.776561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:23.777046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:23.777518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:23.783447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:23.890106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:23.891658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746afbe6-36bd-411a-aa30-89ad91a433d1 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:23.912009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1047/2096] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:29.980986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=77 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:33.896981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:35.548466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:35.563935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3391a281-c3ec-4e5e-a245-4566e80c20fa service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:35.569784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:35.571065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:35.572177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:35.572471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:35.575785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:35.719865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:35.721118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd288be5-0fea-48dc-9720-310400421716 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:35.742482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1050/2097] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:37.074363 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516bf25-a263-44c2-b95a-8a0d2067154d None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.075059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516bf25-a263-44c2-b95a-8a0d2067154d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.075459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9516bf25-a263-44c2-b95a-8a0d2067154d None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.077838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1048/2098] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:20:37.084765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ff18-d0f3-4ea0-9f90-b35153b3334b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.085268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ff18-d0f3-4ea0-9f90-b35153b3334b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.085355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b9ff18-d0f3-4ea0-9f90-b35153b3334b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.150710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-63b9ff18-d0f3-4ea0-9f90-b35153b3334b None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:37.150710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-63b9ff18-d0f3-4ea0-9f90-b35153b3334b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:37.242264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1051/2099] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:20:37.256649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516bf25-a263-44c2-b95a-8a0d2067154d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:37.269800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9516bf25-a263-44c2-b95a-8a0d2067154d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:37.275142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:37.275969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.276318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.276756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.288907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:37.290337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7dee27-19db-481a-98ee-9f9149eae558 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:37.303826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1049/2100] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:37.401952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36098b-4a7e-4d56-8174-7293c82c15ed None None] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.402250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36098b-4a7e-4d56-8174-7293c82c15ed None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.402592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f36098b-4a7e-4d56-8174-7293c82c15ed None None] PATH_INFO: `/` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.404942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1052/2101] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:20:37.417868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e63228-8130-4e22-b7be-84a6860398e2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.418151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e63228-8130-4e22-b7be-84a6860398e2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.418915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e63228-8130-4e22-b7be-84a6860398e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.440437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a5e63228-8130-4e22-b7be-84a6860398e2 None None] MFA Rules not processed for user `9900bcd989014ec783288c63d352d5c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:37.441914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a5e63228-8130-4e22-b7be-84a6860398e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:37.448683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1050/2102] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:20:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:20:37.461961 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36098b-4a7e-4d56-8174-7293c82c15ed None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:37.474754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f36098b-4a7e-4d56-8174-7293c82c15ed None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:37.478514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:37.479188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.479485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.479803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.492836 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:37.495482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951e57d5-f3e8-43f0-9efc-eac708c5cc1f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:37.506036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1053/2103] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:20:37.611496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c367-25f7-4ec5-a5e9-7fb2f553a64e None None] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.612080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c367-25f7-4ec5-a5e9-7fb2f553a64e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.612689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3189c367-25f7-4ec5-a5e9-7fb2f553a64e None None] PATH_INFO: `/` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.614261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1051/2104] 10.222.0.23 () {54 vars in 921 bytes} [Sun Oct 23 02:20:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Oct 23 02:20:37.620581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42dd-a419-4351-974f-f22c4b92ba95 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.621071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42dd-a419-4351-974f-f22c4b92ba95 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.622031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7f42dd-a419-4351-974f-f22c4b92ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.644929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1c7f42dd-a419-4351-974f-f22c4b92ba95 None None] MFA Rules not processed for user `8060598d22754154a1c7af65a96aaac2`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:37.646267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1c7f42dd-a419-4351-974f-f22c4b92ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:37.657175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1054/2105] 10.222.0.23 () {58 vars in 1018 bytes} [Sun Oct 23 02:20:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:20:37.668064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189c367-25f7-4ec5-a5e9-7fb2f553a64e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:37.686342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3189c367-25f7-4ec5-a5e9-7fb2f553a64e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:37.694143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:37.695639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:37.695942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:37.696224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:37.710671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:37.712196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:37.725918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1052/2106] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:20:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:20:42.931411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:42.931892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:42.931991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:42.976233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:42.979159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:42.998237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1055/2107] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:20:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:20:43.013528 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:43.027141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d696b303-ec85-4a22-a3e0-3e4ea81ac96a service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:43.036542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:43.037262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:43.037670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:43.037954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:43.047493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:43.049085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:43.061560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1053/2108] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:20:45.589177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:45.591370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:45.616364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:45.640882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0ff982-9380-480f-ae2b-3a3cf2ef8cc2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:45.653120 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:45.653772 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:45.654055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:45.654331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] PATH_INFO: `/v3/users/102283eee43142f29645e3faa6501891` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:45.655527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=102283eee43142f29645e3faa6501891)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:45.657461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:45.758089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '102283eee43142f29645e3faa6501891'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:45.770074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '102283eee43142f29645e3faa6501891'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:45.783480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '102283eee43142f29645e3faa6501891'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:45.784067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '102283eee43142f29645e3faa6501891'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:45.784948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '102283eee43142f29645e3faa6501891'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:45.818857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1056/2109] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:20:45 2022] DELETE /identity/v3/users/102283eee43142f29645e3faa6501891 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:20:46.093177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:46.108130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257afd9f-33ce-4e54-b990-80bfb8f0a371 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:46.248183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:46.249508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:46.250076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:46.251465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] PATH_INFO: `/v3/projects/d7bb15a157314fb095d2333f64ab0c83` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:46.253804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7bb15a157314fb095d2333f64ab0c83)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:46.256382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:46.327456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7bb15a157314fb095d2333f64ab0c83', 'request_id': 'req-afb035c8-d166-4482-a628-f76fc3d7b827'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.334651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7bb15a157314fb095d2333f64ab0c83', 'request_id': 'req-afb035c8-d166-4482-a628-f76fc3d7b827'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.341128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7bb15a157314fb095d2333f64ab0c83', 'request_id': 'req-afb035c8-d166-4482-a628-f76fc3d7b827'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.357862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] The token cache is being invalidate because project d7bb15a157314fb095d2333f64ab0c83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:20:46.358208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.359165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.367045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1054/2110] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:20:46 2022] DELETE /identity/v3/projects/d7bb15a157314fb095d2333f64ab0c83 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:20:46.371584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:46.373258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:46.393106 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73555a0a-bd6d-4ca3-94db-3927e8aa01a5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:46.471910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:46.472847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:46.473263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:46.473602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] PATH_INFO: `/v3/users/a7e3e6ba208d40b28e6a44a7f61fb1cf` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:46.475135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a7e3e6ba208d40b28e6a44a7f61fb1cf)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:46.477336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:46.554885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a7e3e6ba208d40b28e6a44a7f61fb1cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.571458 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a7e3e6ba208d40b28e6a44a7f61fb1cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.581100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a7e3e6ba208d40b28e6a44a7f61fb1cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.581810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a7e3e6ba208d40b28e6a44a7f61fb1cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.582749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-a956ec4f-9f05-4bc4-a2a9-afbe821095f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a7e3e6ba208d40b28e6a44a7f61fb1cf'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:46.600935 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:46.615588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1057/2111] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:20:46 2022] DELETE /identity/v3/users/a7e3e6ba208d40b28e6a44a7f61fb1cf => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:20:46.976584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:20:46.978133 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:46.991564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb035c8-d166-4482-a628-f76fc3d7b827 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:20:47.055474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:47.080921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:20:47.081654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:47.082008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:47.082358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] PATH_INFO: `/v3/projects/e2c335c445c94b8bbec595e062c254a6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:47.083981 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2c335c445c94b8bbec595e062c254a6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:20:47.092991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:20:47.160780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2c335c445c94b8bbec595e062c254a6', 'request_id': 'req-84aff2f2-6330-40c3-a26a-837b0a58360e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:47.170050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2c335c445c94b8bbec595e062c254a6', 'request_id': 'req-84aff2f2-6330-40c3-a26a-837b0a58360e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:47.176691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2c335c445c94b8bbec595e062c254a6', 'request_id': 'req-84aff2f2-6330-40c3-a26a-837b0a58360e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:47.192701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e3d8-b2c9-47d9-b3a8-a7ff3db0383d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:20:47.193149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e3d8-b2c9-47d9-b3a8-a7ff3db0383d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:20:47.193529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c0e3d8-b2c9-47d9-b3a8-a7ff3db0383d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:20:47.194470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] The token cache is being invalidate because project e2c335c445c94b8bbec595e062c254a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:20:47.194801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:47.195646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:20:47.197434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1055/2112] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:20:46 2022] DELETE /identity/v3/projects/e2c335c445c94b8bbec595e062c254a6 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:20:47.216037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-28c0e3d8-b2c9-47d9-b3a8-a7ff3db0383d None None] MFA Rules not processed for user `66975cf6d8fa417e8df429d4c89ef910`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:20:47.217523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-28c0e3d8-b2c9-47d9-b3a8-a7ff3db0383d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:20:47.307305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1058/2113] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:20:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:20:57.190073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=31,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:20:57.294950 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=1,keystone:SELECT=27 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:06.737637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:06.739193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:06.812314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84aff2f2-6330-40c3-a26a-837b0a58360e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:06.815998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:06.816602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:06.816885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:06.817158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:06.818939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:06.891753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:06.893475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:06.909398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1056/2114] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:06 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:11.769257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:11.769774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:11.769881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:11.794002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:11.795282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:11.876920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1059/2115] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:11.896048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:11.897553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:11.997993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242163a8-4626-445d-8def-ab5f9fa41d97 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:12.003281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:12.003906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:12.004188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:12.004463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:12.015078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:12.016673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:12.025567 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1057/2116] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:21:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:21:17.376639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:17.378111 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:17.390957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f326c186-d2b3-4897-8a9a-49735e8bc407 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:17.394500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:17.395133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:17.395417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:17.395691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:17.397453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:17.469214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:17.470676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:17.482189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1060/2117] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:18.664466 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:18.674582 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3adab0f-8f8d-414a-9a65-ffd5b112c5c0 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:18.678062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:18.678684 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:18.678963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:18.679235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:18.680954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:18.695351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:18.696760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:18.707872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1058/2118] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:20.135579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:20.144963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4b5cc-19e9-461d-810a-ef5798a7467f service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:20.149124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:20.149751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:20.150034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:20.150310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:20.152396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:20.216017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:20.217440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:20.227339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1061/2119] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:23.564882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:23.566623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:23.588568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5d803-a794-4bae-8808-5fc61b1b1b53 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:23.594065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:23.594734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:23.595018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:23.595297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] PATH_INFO: `/v3/users/006623bab8f04f448edf0d43397192d8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:23.611161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] RBAC: Authorizing `identity:delete_user(user_id=006623bab8f04f448edf0d43397192d8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:23.613107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:23.627271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:23.670594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006623bab8f04f448edf0d43397192d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:23.679527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006623bab8f04f448edf0d43397192d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:23.680634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006623bab8f04f448edf0d43397192d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:23.693102 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006623bab8f04f448edf0d43397192d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:23.694098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-8ec6f58f-046b-4fc4-932c-556218cd680e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006623bab8f04f448edf0d43397192d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:23.721768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1059/2120] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:21:23 2022] DELETE /identity/v3/users/006623bab8f04f448edf0d43397192d8 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:23.908737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:23.931742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ec258f-115f-4b14-95ac-d107d716f24e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:23.968179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:24.011620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.013306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.013754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.014197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] PATH_INFO: `/v3/projects/49a39f93e67c4d3f9529bdef5bb66b4c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.025644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a39f93e67c4d3f9529bdef5bb66b4c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.028793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.080455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a39f93e67c4d3f9529bdef5bb66b4c', 'request_id': 'req-213a28f5-d961-4dc2-a7a0-82fa847314b4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:24.088085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a39f93e67c4d3f9529bdef5bb66b4c', 'request_id': 'req-213a28f5-d961-4dc2-a7a0-82fa847314b4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:24.095149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a39f93e67c4d3f9529bdef5bb66b4c', 'request_id': 'req-213a28f5-d961-4dc2-a7a0-82fa847314b4'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:24.111783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] The token cache is being invalidate because project 49a39f93e67c4d3f9529bdef5bb66b4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:21:24.112104 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:24.112897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-213a28f5-d961-4dc2-a7a0-82fa847314b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:24.114782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1062/2121] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:21:23 2022] DELETE /identity/v3/projects/49a39f93e67c4d3f9529bdef5bb66b4c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:24.119458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.120047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.120499 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.177972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:24.179277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:24.254924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1060/2122] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:24.256797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.257082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.257607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.277936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:24.279202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:24.287683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1063/2123] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:24.291727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.302015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd44c4-581a-408b-b060-e4a3b54984a6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.306626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.307230 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.307519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.307796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.308435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:21:24.308769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.310362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.325616 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1061/2124] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:21:24.327093 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.336565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f8a1e-2d1a-4bce-8b65-876abd39168b None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.342400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.343060 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.343326 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.343600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.344237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.346285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.386489 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1064/2125] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/projects => generated 403 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:24.389946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.400006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6419783d-bb04-4938-9da4-7b18cb8ee8e0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.453042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:24.464452 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.466421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.467100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.468165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.470001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.472160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.500371 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1062/2126] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:24.503397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.514982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31ef6ce-d82d-4337-8019-5d665b760dd6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.519629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.520237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.520510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.520779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.521420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.523124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.536952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1065/2127] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:24.538189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.548983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a2976-2032-4920-95ff-4aab88e819d8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.553660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.554260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.554608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.554883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.555522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.557126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bcec26-077b-40dd-baa2-0d02035d9afe admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.569633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1063/2128] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:24.573856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.583937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923fc58-0fce-4cac-87a5-71ea43274ace admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.588511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.589155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.589436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.589835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] PATH_INFO: `/v3/projects/dc8f5c2dccf04b7fa1427b861e2be73e/users/efb17211ebec4f91abc387c37b6c486c/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.606230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc8f5c2dccf04b7fa1427b861e2be73e, user_id=efb17211ebec4f91abc387c37b6c486c, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.608920 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.623435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1066/2129] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:24 2022] PUT /identity/v3/projects/dc8f5c2dccf04b7fa1427b861e2be73e/users/efb17211ebec4f91abc387c37b6c486c/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:24.634426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.634750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.635013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.662927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] MFA Rules not processed for user `efb17211ebec4f91abc387c37b6c486c`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:24.664142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:24.736673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1064/2130] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:24.741174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.775401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:24.807614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc06901a-5832-403e-8d46-21b1db9c9437 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.812270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.812883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.813165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.813440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.814091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.816032 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.846124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1067/2131] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/projects => generated 405 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:24.850388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.862695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b83bc3-5b8b-4939-b5a9-7ae36001903e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.921580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.922191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.922500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.922782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.923453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.925424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.945752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1065/2132] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:24 2022] POST /identity/v3/users => generated 309 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:24.949997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.960656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96822431-eef9-431b-855f-87b11426a2cf admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:24.966433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:24.967064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:24.967335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:24.967719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:24.968357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:24.969946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:24.981259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1068/2133] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:21:24.985628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:24.995523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ae95a0-e38b-4c0d-b70b-516feb16d411 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.000711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.001398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.001681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.001957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] PATH_INFO: `/v3/projects/4e35f9b32f1d4426836af239d377c7eb/users/5cc6c1e76d234f928c5d324c9c2aa213/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.017043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e35f9b32f1d4426836af239d377c7eb, user_id=5cc6c1e76d234f928c5d324c9c2aa213, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.019820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.034292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1066/2134] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:24 2022] PUT /identity/v3/projects/4e35f9b32f1d4426836af239d377c7eb/users/5cc6c1e76d234f928c5d324c9c2aa213/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:25.038373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:25.049185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd39d9-31ff-4d5f-9a02-c05f52560743 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.104373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.104972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.105267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.106174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.106816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.108418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddfe4b5-d98c-468e-9098-8576588d8bd0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.117796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1069/2135] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:25.121901 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:25.131900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2b495e-3fa3-4fbf-80c8-90acaa8d3933 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.137727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.138853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.139232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.139515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] PATH_INFO: `/v3/projects/4e35f9b32f1d4426836af239d377c7eb/users/5cc6c1e76d234f928c5d324c9c2aa213/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.141423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e35f9b32f1d4426836af239d377c7eb, user_id=5cc6c1e76d234f928c5d324c9c2aa213, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.144086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.154842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1067/2136] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:25 2022] PUT /identity/v3/projects/4e35f9b32f1d4426836af239d377c7eb/users/5cc6c1e76d234f928c5d324c9c2aa213/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:25.164955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.165245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.165523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.192516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] MFA Rules not processed for user `5cc6c1e76d234f928c5d324c9c2aa213`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:25.193709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:25.262164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1070/2137] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:25 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:25.272795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:25.274536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:25.289838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:25.332226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df92f37-f37a-4fa4-a8e3-34542c6421a1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.335885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.336487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.336770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.337046 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.346605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.348033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca67bd41-cce8-457e-bfc1-03973a0c58e2 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.357454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1068/2138] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:25 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:25.519140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:25.550857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:25.581210 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c05f4b-4e20-402a-86b1-c77429014195 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.585671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.586285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.586591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.586864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] PATH_INFO: `/v3/users/efb17211ebec4f91abc387c37b6c486c` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.587949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb17211ebec4f91abc387c37b6c486c)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.589844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.636051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb17211ebec4f91abc387c37b6c486c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:25.643567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb17211ebec4f91abc387c37b6c486c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:25.650699 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb17211ebec4f91abc387c37b6c486c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:25.651320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb17211ebec4f91abc387c37b6c486c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:25.651927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb17211ebec4f91abc387c37b6c486c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:25.670534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1071/2139] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:21:25 2022] DELETE /identity/v3/users/efb17211ebec4f91abc387c37b6c486c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:25.675461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.675959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.676064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.695226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:25.696435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:25.754267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1069/2140] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:25.764209 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:25.765594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:25.822713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1171bc-3838-4c7e-9930-656573ac7054 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:25.826656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:25.827251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:25.827531 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:25.827806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:25.836675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:25.838096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:25.847999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1072/2141] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:26.114685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:26.116138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:26.128003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fa3636-1b67-4100-8f19-e0738fdeca0f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:26.132986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:26.133614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.133896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.134167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] PATH_INFO: `/v3/projects/dc8f5c2dccf04b7fa1427b861e2be73e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.135318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc8f5c2dccf04b7fa1427b861e2be73e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:26.137769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:26.183892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc8f5c2dccf04b7fa1427b861e2be73e', 'request_id': 'req-9d395c34-3120-4712-a3b0-8b9a73d2f494'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.190061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc8f5c2dccf04b7fa1427b861e2be73e', 'request_id': 'req-9d395c34-3120-4712-a3b0-8b9a73d2f494'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.195816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc8f5c2dccf04b7fa1427b861e2be73e', 'request_id': 'req-9d395c34-3120-4712-a3b0-8b9a73d2f494'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.208960 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] The token cache is being invalidate because project dc8f5c2dccf04b7fa1427b861e2be73e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:21:26.209244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.209856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.211387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1070/2142] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:21:26 2022] DELETE /identity/v3/projects/dc8f5c2dccf04b7fa1427b861e2be73e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:26.216705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:26.226379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac1d846-5a9d-4628-9170-68745fcd8251 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:26.275006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:26.287625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:26.288274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.288551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.288821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] PATH_INFO: `/v3/users/5cc6c1e76d234f928c5d324c9c2aa213` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.289919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5cc6c1e76d234f928c5d324c9c2aa213)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:26.291847 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:26.337134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5cc6c1e76d234f928c5d324c9c2aa213'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.343998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5cc6c1e76d234f928c5d324c9c2aa213'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.351344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5cc6c1e76d234f928c5d324c9c2aa213'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.351946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5cc6c1e76d234f928c5d324c9c2aa213'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.352659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-2095f80e-88b3-4fb4-98d1-e6413622fe60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5cc6c1e76d234f928c5d324c9c2aa213'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.370190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1073/2143] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:21:26 2022] DELETE /identity/v3/users/5cc6c1e76d234f928c5d324c9c2aa213 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:21:26.648406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:26.649833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:26.663116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d395c34-3120-4712-a3b0-8b9a73d2f494 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:26.705379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:26.724965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:26.725595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.725876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.726151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] PATH_INFO: `/v3/projects/4e35f9b32f1d4426836af239d377c7eb` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.727459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e35f9b32f1d4426836af239d377c7eb)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:26.729219 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:26.774372 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e35f9b32f1d4426836af239d377c7eb', 'request_id': 'req-3efd3fc6-0bad-4769-b056-4014fa51d317'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.780519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e35f9b32f1d4426836af239d377c7eb', 'request_id': 'req-3efd3fc6-0bad-4769-b056-4014fa51d317'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.785910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e35f9b32f1d4426836af239d377c7eb', 'request_id': 'req-3efd3fc6-0bad-4769-b056-4014fa51d317'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.798748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] The token cache is being invalidate because project 4e35f9b32f1d4426836af239d377c7eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:21:26.799037 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.799647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3efd3fc6-0bad-4769-b056-4014fa51d317 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:26.801140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1071/2144] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:21:26 2022] DELETE /identity/v3/projects/4e35f9b32f1d4426836af239d377c7eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:26.806180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.806501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.806778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.826296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:26.827630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:26.885751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1074/2145] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:26.894993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.895216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.895492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.915751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:26.916948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:26.925535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1072/2146] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:26.929704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:26.939998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c865ebe-731b-42d7-857f-175a0302d522 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:26.944597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:26.945193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.945480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.945757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.946564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:21:26.946729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:26.948336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:26.959928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1075/2147] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:21:26.964088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:26.973130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28de5b-93fd-4ad5-b8f6-f0afea44190b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:26.977715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:26.978299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:26.978618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:26.978899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:26.979518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:26.982004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.008398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1073/2148] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:26 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:27.012954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:27.022829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc7a87-fa5f-44d4-a0ae-6caf29137427 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:27.073019 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:27.079584 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:27.080196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.080758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.081085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.081703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:27.083933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.105107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1076/2149] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:27 2022] POST /identity/v3/users => generated 303 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:27.109840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:27.119918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f44578-52b1-46b3-b3ad-570c9a0bed53 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:27.124579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:27.125155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.125464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.125739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.126385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:27.127991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.139333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1074/2150] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:27.143312 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:27.153830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876ead58-b539-4270-86cd-e21d678ae965 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:27.158342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:27.158968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.159250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.159544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.160163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:27.161753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f40377-e1ac-4114-9839-707c898f26a2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.174247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1077/2151] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:21:27.178264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:27.188004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863c727-de0b-478c-94a8-4b7d70bcde2d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:27.192572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:27.193212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.193491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.193763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] PATH_INFO: `/v3/projects/c6a189e26117404f9663c6e666c02dea/users/2cb7135cb94f40de91e682c656644e14/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.207917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6a189e26117404f9663c6e666c02dea, user_id=2cb7135cb94f40de91e682c656644e14, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:27.210571 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.224000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1075/2152] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:27 2022] PUT /identity/v3/projects/c6a189e26117404f9663c6e666c02dea/users/2cb7135cb94f40de91e682c656644e14/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:21:27.229003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e27d8-cc33-4cbb-ae4e-ba337c714196 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.229714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e27d8-cc33-4cbb-ae4e-ba337c714196 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.229714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e27d8-cc33-4cbb-ae4e-ba337c714196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.249076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7c5e27d8-cc33-4cbb-ae4e-ba337c714196 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:27.250276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7c5e27d8-cc33-4cbb-ae4e-ba337c714196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:27.307607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1078/2153] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:27.317319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:27.319029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:27.376249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59128ce9-3cca-44d6-9980-2b0dabb13d5f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:27.379947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:27.380545 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:27.380822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:27.381091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:27.391588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:27.393109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:27.403817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1076/2154] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:28.044406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.045433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.045774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.078465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] MFA Rules not processed for user `2cb7135cb94f40de91e682c656644e14`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:28.079798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:28.148370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1079/2155] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:28 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:28.153343 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.162505 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4715c-1fb6-49c1-aed8-11729225829e service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.167018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.167636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.168431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.168713 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.169354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.171285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.176630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:28.201704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1077/2156] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:28 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:21:28.205580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.216274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053a5e9f-d01d-41c1-b5a2-65546fa7b1df None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.272938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.273678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.273811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.274079 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.274734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.276748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.287571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:28.299883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1080/2157] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:28 2022] POST /identity/v3/users => generated 309 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:21:28.303661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.314390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9621ae3d-cc33-4eaf-a231-a2e5bbb30b68 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.319033 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.319630 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.319911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.320184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.320824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.322414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.332229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1078/2158] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:28.336296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.346379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27d5650-efd7-4a00-b8e2-f67fbda5cb95 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.350956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.351592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.351875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.352149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] PATH_INFO: `/v3/projects/819188acf6064e4b9609515f64febc40/users/ea1e7b187a6443ba85e508513f1fd5c3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.367738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819188acf6064e4b9609515f64febc40, user_id=ea1e7b187a6443ba85e508513f1fd5c3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.370396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.383716 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1081/2159] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:28 2022] PUT /identity/v3/projects/819188acf6064e4b9609515f64febc40/users/ea1e7b187a6443ba85e508513f1fd5c3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:28.387916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.397624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481287d-a3f8-4049-9ddc-fb68e3838db7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.456864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.457381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.457659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.457929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.458590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.460177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb460fbc-c8e7-4d49-ac3e-f9bdbbb31582 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.469900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1079/2160] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:28.474068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:28.484864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666c449-f4d9-4eda-ac16-4a78cff641c4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:28.489379 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:28.490015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:28.490292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:28.490668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] PATH_INFO: `/v3/projects/819188acf6064e4b9609515f64febc40/users/ea1e7b187a6443ba85e508513f1fd5c3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:28.492693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=819188acf6064e4b9609515f64febc40, user_id=ea1e7b187a6443ba85e508513f1fd5c3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:28.495697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:28.507155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1082/2161] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:28 2022] PUT /identity/v3/projects/819188acf6064e4b9609515f64febc40/users/ea1e7b187a6443ba85e508513f1fd5c3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:29.123427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:29.123722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:29.124041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:29.154893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] MFA Rules not processed for user `ea1e7b187a6443ba85e508513f1fd5c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:29.156128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:29.229657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1080/2162] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:29 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:29.837930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:29.839751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:29.917244 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09db036-28f7-493f-83bf-98bbc7025140 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:29.923051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:29.923657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:29.923942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:29.924218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:29.926133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:29.941889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:29.943017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:29.954216 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1083/2163] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:29 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:21:30.444967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:30.502402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:30.507215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b790fa-b4be-43a6-87fa-00dc43838f37 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:30.513099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:30.513754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:30.514040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:30.514316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] PATH_INFO: `/v3/users/2cb7135cb94f40de91e682c656644e14` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:30.515460 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cb7135cb94f40de91e682c656644e14)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:30.517399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:30.561386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cb7135cb94f40de91e682c656644e14'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.568298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cb7135cb94f40de91e682c656644e14'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.568965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cb7135cb94f40de91e682c656644e14'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.575985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cb7135cb94f40de91e682c656644e14'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.576649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cb7135cb94f40de91e682c656644e14'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.595485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1081/2164] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:21:30 2022] DELETE /identity/v3/users/2cb7135cb94f40de91e682c656644e14 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:30.766301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:30.777858 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04b978-766a-41c2-88c0-8237f6659447 service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:30.834864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:30.835496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:30.835803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:30.836078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] PATH_INFO: `/v3/projects/c6a189e26117404f9663c6e666c02dea` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:30.837187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6a189e26117404f9663c6e666c02dea)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:30.839133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:30.883810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6a189e26117404f9663c6e666c02dea', 'request_id': 'req-84c9bfbe-d592-4060-9d1a-0752288593c5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.884865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:30.892007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6a189e26117404f9663c6e666c02dea', 'request_id': 'req-84c9bfbe-d592-4060-9d1a-0752288593c5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.897972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6a189e26117404f9663c6e666c02dea', 'request_id': 'req-84c9bfbe-d592-4060-9d1a-0752288593c5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.909533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] The token cache is being invalidate because project c6a189e26117404f9663c6e666c02dea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:21:30.909819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.910453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:30.911930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1084/2165] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:21:30 2022] DELETE /identity/v3/projects/c6a189e26117404f9663c6e666c02dea => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:30.916035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:30.917409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:30.930414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10ab32c-78b2-4d1c-b659-353d87a01183 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:30.986099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:30.986751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:30.987032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:30.987302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] PATH_INFO: `/v3/users/ea1e7b187a6443ba85e508513f1fd5c3` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:30.988415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea1e7b187a6443ba85e508513f1fd5c3)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:30.990317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.035892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea1e7b187a6443ba85e508513f1fd5c3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.043579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea1e7b187a6443ba85e508513f1fd5c3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.044446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea1e7b187a6443ba85e508513f1fd5c3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.051627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea1e7b187a6443ba85e508513f1fd5c3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.052243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b4271f01-afbc-448c-8b2a-70a12918f10e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea1e7b187a6443ba85e508513f1fd5c3'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.063118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:31.070348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1082/2166] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:21:30 2022] DELETE /identity/v3/users/ea1e7b187a6443ba85e508513f1fd5c3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:31.238876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.240391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:31.254549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c9bfbe-d592-4060-9d1a-0752288593c5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.312029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.312657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.312937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.313207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] PATH_INFO: `/v3/projects/819188acf6064e4b9609515f64febc40` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.314303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=819188acf6064e4b9609515f64febc40)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.316296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.362394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '819188acf6064e4b9609515f64febc40', 'request_id': 'req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.368428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '819188acf6064e4b9609515f64febc40', 'request_id': 'req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.374287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '819188acf6064e4b9609515f64febc40', 'request_id': 'req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.387292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] The token cache is being invalidate because project 819188acf6064e4b9609515f64febc40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:21:31.387705 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.388298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-9baa072a-c935-4f84-a694-ebbf2ed9fc7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:21:31.389738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1085/2167] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:21:31 2022] DELETE /identity/v3/projects/819188acf6064e4b9609515f64febc40 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:31.394657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.394953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.395228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.416331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:31.417479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:31.475412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1083/2168] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:31.483119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.483406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.483691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.504221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:31.505418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:31.514068 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1086/2169] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:31.518670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.528388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39741af4-2f64-4094-bd27-3fe6a762ad61 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.532972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.533568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.533853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.534131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.534807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:21:31.535119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.536717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.548337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1084/2170] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:21:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:21:31.552995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.562972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdab9112-aa34-4282-a168-d64fa6b3f4ec None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.567604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b912-459e-402f-9312-048817c76832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.568197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b912-459e-402f-9312-048817c76832 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.568478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b912-459e-402f-9312-048817c76832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.568753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520b912-459e-402f-9312-048817c76832 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.569380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520b912-459e-402f-9312-048817c76832 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.571351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520b912-459e-402f-9312-048817c76832 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.598514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1087/2171] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:31.602825 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.612890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ee0084-7aa6-4dee-8241-b2748f54a619 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.668421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.668951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.669238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.669516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.670151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.672071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.694585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1085/2172] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:21:31.698906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b912-459e-402f-9312-048817c76832 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.714199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520b912-459e-402f-9312-048817c76832 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.719538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.720134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.720410 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.720677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.721309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.722932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.733857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1088/2173] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:31.738005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.748552 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b4a32a-af86-42d3-b85d-cc71000e8579 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.753205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.753803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.754084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.754364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.755043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.756897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e254bf-34b4-4936-b9d3-45c3846b7e4c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.768045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1086/2174] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:21:31.772188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.782175 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1ca200-a305-4138-8bba-530060006de0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.787053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.787743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.788026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.788302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] PATH_INFO: `/v3/projects/d25f63f883bf4a9eb9c899f0b1e25043/users/5085390989104f859e7f27456eeae755/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.802668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d25f63f883bf4a9eb9c899f0b1e25043, user_id=5085390989104f859e7f27456eeae755, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:31.805302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:31.818139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1089/2175] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:31 2022] PUT /identity/v3/projects/d25f63f883bf4a9eb9c899f0b1e25043/users/5085390989104f859e7f27456eeae755/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:21:31.829025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.829320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.829601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.857543 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] MFA Rules not processed for user `5085390989104f859e7f27456eeae755`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:31.858778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:31.905446 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:31.923143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1087/2176] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:31.927639 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:31.934384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:31.990924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30029ab4-bc34-4d68-87ec-b20abb400dd7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:31.995666 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:31.996933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:31.997211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:31.997510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:31.998131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.000067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.025972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1090/2177] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:21:31 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:21:32.030148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.040759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fe954f-481a-4c7b-bbd0-0a4522af9110 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.097437 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.098044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.098328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.098645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.099272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.101188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.122793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1088/2178] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:21:32 2022] POST /identity/v3/users => generated 295 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:21:32.127030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.138027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893a9ac-0292-4f1f-8b15-a2d8d53000a6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.142927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.143513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.143790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.144057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.144681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.146267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.157547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1091/2179] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:21:32.161729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.171948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a7a1c5-4e5b-4044-b548-247ca3860bea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.176641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.177223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.177506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.177782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] PATH_INFO: `/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e/users/5a255d3ed118438a924d07d7064e75a3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.192309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaccd2ad6b304bd5b6cfba5bffb99b8e, user_id=5a255d3ed118438a924d07d7064e75a3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.195121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.208560 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1089/2180] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:32 2022] PUT /identity/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e/users/5a255d3ed118438a924d07d7064e75a3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:32.212581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.223136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3411f597-c6ef-4bc5-b9b9-8eabb76b72c7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.278571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.279172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.279448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.279720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.280343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.281929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f94826-daf0-4852-8b4b-3e6c17d7fc4f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.291864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1092/2181] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:21:32 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:21:32.297085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.309592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e078bc-a87e-4e76-babe-c64a4de6dd23 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.314222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.314905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.315173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.315447 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] PATH_INFO: `/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e/users/5a255d3ed118438a924d07d7064e75a3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.317334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaccd2ad6b304bd5b6cfba5bffb99b8e, user_id=5a255d3ed118438a924d07d7064e75a3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.319994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.330968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1090/2182] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:21:32 2022] PUT /identity/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e/users/5a255d3ed118438a924d07d7064e75a3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:21:32.343085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.343369 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.343650 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.371546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] MFA Rules not processed for user `5a255d3ed118438a924d07d7064e75a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:21:32.372766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:32.437795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:32.441771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1093/2183] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:21:32 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:21:32.453236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:32.455013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:32.514065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80dd7da-69c0-4b07-841e-98e5d64edf19 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:32.517883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:32.518514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:32.518799 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:32.519074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:32.574564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:32.576032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:32.579520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:32.586286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1091/2184] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:39.277684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:39.287234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f63f08-a4f7-459a-aaaf-d1218426d7a3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:39.290764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:39.291351 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:39.291627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:39.291893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:39.301133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:39.302570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:39.312590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1094/2185] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:39 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:42.587224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:43.698773 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:21:43.700335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:43.767642 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02880c34-6227-4587-84ba-f9e6c8a3bb27 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:21:43.771190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:21:43.771834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:21:43.772123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:21:43.772402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:21:43.774134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:21:43.784351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:21:43.785785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010f10d1-de0c-427d-ae26-022122dae3b6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:21:43.794612 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1092/2186] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:21:43 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:21:49.307669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:21:53.790084 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=26 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:08.317674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:08.378678 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f988529-82cb-4900-a882-7edf7de1f083 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:08.383448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:08.384062 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:08.384343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:08.384619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] PATH_INFO: `/v3/users/5085390989104f859e7f27456eeae755` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:08.385719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5085390989104f859e7f27456eeae755)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:08.387764 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:08.433087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5085390989104f859e7f27456eeae755'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.440052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5085390989104f859e7f27456eeae755'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.447085 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5085390989104f859e7f27456eeae755'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.447698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5085390989104f859e7f27456eeae755'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.448303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5085390989104f859e7f27456eeae755'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.466122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1095/2187] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:08 2022] DELETE /identity/v3/users/5085390989104f859e7f27456eeae755 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:08.470780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:08.471278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:08.471379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:08.491249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:08.492431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:08.549468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1093/2188] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:08.559420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:08.560805 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:08.617592 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65990e49-390c-4799-a788-802828c0da2e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:08.621993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:08.622414 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:08.622724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:08.622992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:08.632074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:08.633493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:08.642718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1096/2189] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:08.909503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:08.910948 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:08.923820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe56d81-4830-4041-a853-ed679c46f2ec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:08.928784 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:08.929421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:08.929699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:08.929972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] PATH_INFO: `/v3/projects/d25f63f883bf4a9eb9c899f0b1e25043` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:08.931117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d25f63f883bf4a9eb9c899f0b1e25043)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:08.933056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:08.979954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd25f63f883bf4a9eb9c899f0b1e25043', 'request_id': 'req-5311135f-1e96-4694-8052-3b4918cfc0d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.987089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd25f63f883bf4a9eb9c899f0b1e25043', 'request_id': 'req-5311135f-1e96-4694-8052-3b4918cfc0d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:08.992647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd25f63f883bf4a9eb9c899f0b1e25043', 'request_id': 'req-5311135f-1e96-4694-8052-3b4918cfc0d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.006296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] The token cache is being invalidate because project d25f63f883bf4a9eb9c899f0b1e25043 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:09.006624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.007256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.008748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1094/2190] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:08 2022] DELETE /identity/v3/projects/d25f63f883bf4a9eb9c899f0b1e25043 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:09.012875 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.022799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1c413-8a8e-4e68-ac6f-b1c938da72f2 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.041047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:09.082502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.083119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.083393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.083660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] PATH_INFO: `/v3/users/5a255d3ed118438a924d07d7064e75a3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.084809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a255d3ed118438a924d07d7064e75a3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.086674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:09.131537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a255d3ed118438a924d07d7064e75a3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.138724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a255d3ed118438a924d07d7064e75a3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.146341 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a255d3ed118438a924d07d7064e75a3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.146995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a255d3ed118438a924d07d7064e75a3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.147589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-589879b7-a45e-4bbf-8a41-5f67d1558b03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a255d3ed118438a924d07d7064e75a3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.165683 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1097/2191] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:09 2022] DELETE /identity/v3/users/5a255d3ed118438a924d07d7064e75a3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:09.499050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.501309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:09.516602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5311135f-1e96-4694-8052-3b4918cfc0d8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.576306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.576935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.577361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.577473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] PATH_INFO: `/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.578640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaccd2ad6b304bd5b6cfba5bffb99b8e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.580660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:09.611244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:09.629205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaccd2ad6b304bd5b6cfba5bffb99b8e', 'request_id': 'req-f7139a41-5316-4b33-aa29-f4a7eb178750'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.635655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaccd2ad6b304bd5b6cfba5bffb99b8e', 'request_id': 'req-f7139a41-5316-4b33-aa29-f4a7eb178750'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.641747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaccd2ad6b304bd5b6cfba5bffb99b8e', 'request_id': 'req-f7139a41-5316-4b33-aa29-f4a7eb178750'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.656670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] The token cache is being invalidate because project aaccd2ad6b304bd5b6cfba5bffb99b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:09.657112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.657737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f7139a41-5316-4b33-aa29-f4a7eb178750 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:09.659330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1095/2192] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:09 2022] DELETE /identity/v3/projects/aaccd2ad6b304bd5b6cfba5bffb99b8e => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:09.664226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.664713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.664821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.691436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:09.692751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:09.750923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1098/2193] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:22:09.758801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.759153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.759508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.779517 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:09.780696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:09.791096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1096/2194] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:09.796030 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.805641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e48078-aac9-4c64-b28b-1e497513a9e5 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.810572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.811166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.811452 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.811728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.812396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:22:09.812698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.814333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:09.825877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1099/2195] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:22:09.830907 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.840769 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d2c373-db73-4e09-84f8-7e7bd4a80be5 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.845429 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.846023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.846303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.846613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.847248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.849138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:09.875890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1097/2196] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:09 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:09.880229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.890626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba04ec30-189a-4ad0-a0fb-08d2010d6a49 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.915996 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:09.947591 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.948189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.948471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.948745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.949405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.951427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:09.973036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1100/2197] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:09 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:09.977183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:09.988030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95b82d-ce95-443e-98c3-2f21ac8e8f53 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:09.992635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:09.993228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:09.993511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:09.993789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:09.994427 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:09.996083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.006104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1098/2198] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:10.010052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.020979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc90d868-2385-4303-a19d-d73c1196ebec admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.025511 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.026089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.026367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.026665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.028272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.029861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3caa0d-e74f-4e68-8a90-c3c587c3b36b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.040163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1101/2199] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:10.044086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.053877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4313364-d598-41a4-879a-16f48e0f2852 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.058410 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.059067 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.059350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.059624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] PATH_INFO: `/v3/projects/a8dbf56822f6411ca629d9a65f3b8181/users/7c14352e30f84635952737b1858ddbcf/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.073623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8dbf56822f6411ca629d9a65f3b8181, user_id=7c14352e30f84635952737b1858ddbcf, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.076956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.089234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1099/2200] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:10 2022] PUT /identity/v3/projects/a8dbf56822f6411ca629d9a65f3b8181/users/7c14352e30f84635952737b1858ddbcf/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:10.102661 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.102946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.103219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.130807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] MFA Rules not processed for user `7c14352e30f84635952737b1858ddbcf`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:10.131917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:10.194945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1102/2201] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:10.200043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.259840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9052af99-8c41-4636-b9ea-4a52cc8145f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.264932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.265518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.265794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.266064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.266737 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.268597 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.294530 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1100/2202] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:10.298514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.309131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcf6964-5aee-463d-a37d-e373f4d64fda None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.363290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.363886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.364163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.364433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.365054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.366972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.388815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1103/2203] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/users => generated 294 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:10.392781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.403032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f3607b-b653-4e88-a35a-655bbc45a250 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.407589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.408179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.408464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.408738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.409376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.410990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.421389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1101/2204] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:10.425287 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.435485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174c561f-0966-4bbb-a2cb-1f0cb2677732 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.439946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.440521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.440799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.441069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.441692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.443305 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665dd092-a592-4312-9fbd-ae6f30862a7c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.453562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1104/2205] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:10.457442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.467531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3d0bcc-a174-49c6-83b1-8576162a44fa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.472441 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.473103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.473478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.473743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] PATH_INFO: `/v3/projects/bbd754fa75654a10895182297ff48a93/users/b902ecd2c93147ce941bc42155a82c82/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.489040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd754fa75654a10895182297ff48a93, user_id=b902ecd2c93147ce941bc42155a82c82, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.491748 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.502492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:10.507445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1102/2206] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:10 2022] PUT /identity/v3/projects/bbd754fa75654a10895182297ff48a93/users/b902ecd2c93147ce941bc42155a82c82/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:10.519415 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.520132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.520481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.550022 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] MFA Rules not processed for user `b902ecd2c93147ce941bc42155a82c82`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:10.551286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:10.555332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:10.619741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1105/2207] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:10.624752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.689420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650a1351-e973-429c-b74f-d0f038147242 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.694027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.694653 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.694932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.695201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.695836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.697751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.725533 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1103/2208] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:10.729581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.740139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d180b2c-84de-4791-aebc-e207f24a48ed None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.797153 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.797748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.798028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.798296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.798949 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.800843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.824735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1106/2209] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:10 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:10.826021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.835928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05fde28-037e-4e26-8178-b20787d6a3d6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.840984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.841550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.841828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.842101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.842780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.844348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.856563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1104/2210] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:22:10.860503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.870151 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58c0749-6702-4aa2-adc8-f292948ef136 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.875355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.875995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.876289 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.876663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] PATH_INFO: `/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5/users/e3bfd317813b4d7ab3f95bef7dcad702/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.893016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a26b8286b5444c8a9f1b57b2f072e5, user_id=e3bfd317813b4d7ab3f95bef7dcad702, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.895689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.908613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1107/2211] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:10 2022] PUT /identity/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5/users/e3bfd317813b4d7ab3f95bef7dcad702/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:10.912670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:10.923081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0eeb56-1cf7-466f-a70c-ada605649010 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:10.977234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:10.977762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:10.978044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:10.978318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:10.978995 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:10.981490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b80508-8962-4b4d-b764-c5a71bbfd32a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:10.990988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1105/2212] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:10 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:10.995134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:11.006114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baccfb3e-5fb8-496d-ae70-0c06b3bd46b3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:11.010698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:11.011459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:11.011618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:11.011895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] PATH_INFO: `/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5/users/e3bfd317813b4d7ab3f95bef7dcad702/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:11.013830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8a26b8286b5444c8a9f1b57b2f072e5, user_id=e3bfd317813b4d7ab3f95bef7dcad702, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:11.016499 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:11.026371 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1108/2213] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:10 2022] PUT /identity/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5/users/e3bfd317813b4d7ab3f95bef7dcad702/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:11.036359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:11.036650 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:11.036926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:11.065080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] MFA Rules not processed for user `e3bfd317813b4d7ab3f95bef7dcad702`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:11.066271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:11.125665 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:11.135849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1106/2214] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:11.146354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:11.148097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:11.207586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd72c4b-cd0a-4031-a541-ee129b40411d admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:11.213087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:11.214112 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:11.215166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:11.215662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:11.224887 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:11.226322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:11.232227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:11.238217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1109/2215] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:11.726387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:11.727858 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:11.740087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b75e6f-5793-473d-9dd9-e7afba01fb2c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:11.743518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:11.744103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:11.744379 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:11.744647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:11.805565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:11.807502 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:11.817151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1107/2216] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:11 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:15.831017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:15.843261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce1b87c-9874-4ca0-91c8-87358dfd16b6 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:15.846923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:15.847544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:15.847844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:15.848122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:15.904357 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:15.905784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5db7151-8bd9-4950-a031-014f5e3fecca service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:15.914532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1110/2217] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:15 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:18.012980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:18.076335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74f2d92-62eb-4592-8c1f-16ed2751071a service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:18.081140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:18.081758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:18.082040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:18.082314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] PATH_INFO: `/v3/users/7c14352e30f84635952737b1858ddbcf` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:18.083458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c14352e30f84635952737b1858ddbcf)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:18.085352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:18.130166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c14352e30f84635952737b1858ddbcf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.137865 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c14352e30f84635952737b1858ddbcf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.138554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c14352e30f84635952737b1858ddbcf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.145675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c14352e30f84635952737b1858ddbcf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.146294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c14352e30f84635952737b1858ddbcf'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.163381 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1108/2218] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:18 2022] DELETE /identity/v3/users/7c14352e30f84635952737b1858ddbcf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:18.168091 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:18.168590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:18.168837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:18.188354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:18.189547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:18.248668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1111/2219] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:18.258610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:18.260051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:18.274544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:18.317898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731ff0d3-ebd3-441f-9008-47a63eba0f2d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:18.321733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:18.322335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:18.322740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:18.323018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:18.332227 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:18.333637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:18.342980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1109/2220] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:18.611304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:18.612797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:18.626803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ae072a-1e3f-4eee-927a-58ebc2265be9 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:18.631634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:18.632825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:18.633298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:18.633579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] PATH_INFO: `/v3/projects/a8dbf56822f6411ca629d9a65f3b8181` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:18.634814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8dbf56822f6411ca629d9a65f3b8181)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:18.636608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:18.683457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8dbf56822f6411ca629d9a65f3b8181', 'request_id': 'req-90d88859-a0e2-41a1-bcbe-d85315c86a8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.690494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8dbf56822f6411ca629d9a65f3b8181', 'request_id': 'req-90d88859-a0e2-41a1-bcbe-d85315c86a8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.696226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8dbf56822f6411ca629d9a65f3b8181', 'request_id': 'req-90d88859-a0e2-41a1-bcbe-d85315c86a8f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.709874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] The token cache is being invalidate because project a8dbf56822f6411ca629d9a65f3b8181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:18.710163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.710823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.712293 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1112/2221] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:18 2022] DELETE /identity/v3/projects/a8dbf56822f6411ca629d9a65f3b8181 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:18.716795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:18.726368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b472f5b-608a-4998-b27c-f6ea013eb356 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:18.781223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:18.781993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:18.782273 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:18.782592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] PATH_INFO: `/v3/users/b902ecd2c93147ce941bc42155a82c82` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:18.783715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b902ecd2c93147ce941bc42155a82c82)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:18.785695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:18.829969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b902ecd2c93147ce941bc42155a82c82'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.836892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b902ecd2c93147ce941bc42155a82c82'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.837537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b902ecd2c93147ce941bc42155a82c82'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.845358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b902ecd2c93147ce941bc42155a82c82'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.846002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b902ecd2c93147ce941bc42155a82c82'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:18.865179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1110/2222] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:18 2022] DELETE /identity/v3/users/b902ecd2c93147ce941bc42155a82c82 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:19.146557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:19.148358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:19.162526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d88859-a0e2-41a1-bcbe-d85315c86a8f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:19.191094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:19.238224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:19.238895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:19.239180 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:19.239456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] PATH_INFO: `/v3/projects/bbd754fa75654a10895182297ff48a93` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:19.240574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd754fa75654a10895182297ff48a93)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:19.242501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:19.288630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd754fa75654a10895182297ff48a93', 'request_id': 'req-d75adba6-39a9-495d-b70f-9089cc324fd0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.295045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd754fa75654a10895182297ff48a93', 'request_id': 'req-d75adba6-39a9-495d-b70f-9089cc324fd0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.300587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd754fa75654a10895182297ff48a93', 'request_id': 'req-d75adba6-39a9-495d-b70f-9089cc324fd0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.312662 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] The token cache is being invalidate because project bbd754fa75654a10895182297ff48a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:19.312954 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.313575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.315074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1113/2223] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:19 2022] DELETE /identity/v3/projects/bbd754fa75654a10895182297ff48a93 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:19.319047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:19.320487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:19.333477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36be74-8f51-4554-8b73-515f4856d5a6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:19.365002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:19.392163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:19.392794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:19.393076 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:19.393354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] PATH_INFO: `/v3/users/e3bfd317813b4d7ab3f95bef7dcad702` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:19.394578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3bfd317813b4d7ab3f95bef7dcad702)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:19.396462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:19.441506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3bfd317813b4d7ab3f95bef7dcad702'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.448450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3bfd317813b4d7ab3f95bef7dcad702'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.449977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3bfd317813b4d7ab3f95bef7dcad702'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.457050 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3bfd317813b4d7ab3f95bef7dcad702'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.457700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-05bd7585-3e01-45df-8b1f-78fa4e2e949c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3bfd317813b4d7ab3f95bef7dcad702'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.474953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1111/2224] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:19 2022] DELETE /identity/v3/users/e3bfd317813b4d7ab3f95bef7dcad702 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:19.766945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:19.768520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:19.781539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75adba6-39a9-495d-b70f-9089cc324fd0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:19.842189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:19.842780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:19.843064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:19.843340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] PATH_INFO: `/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:19.844502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8a26b8286b5444c8a9f1b57b2f072e5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:19.846519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:19.897906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8a26b8286b5444c8a9f1b57b2f072e5', 'request_id': 'req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.905002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8a26b8286b5444c8a9f1b57b2f072e5', 'request_id': 'req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.906187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:19.911965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8a26b8286b5444c8a9f1b57b2f072e5', 'request_id': 'req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.924185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] The token cache is being invalidate because project a8a26b8286b5444c8a9f1b57b2f072e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:19.924486 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.925095 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ae9a90a9-c58a-47d7-8a89-bc8c797f823b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:19.926646 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1114/2225] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:19 2022] DELETE /identity/v3/projects/a8a26b8286b5444c8a9f1b57b2f072e5 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:19.932490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:19.932810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:19.933610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:19.954625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:19.955857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.014501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1112/2226] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:20.022952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.023241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.023523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.043572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:20.044771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.052549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1115/2227] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:20.056629 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.066167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6f3d9f-8c8a-47a5-bb0a-7e9786e1bb3f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.070686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.071258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.071535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.071805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.072428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:22:20.072727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.074934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.086186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1113/2228] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:22:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:22:20.091177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.100233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff1f38f-7151-4d2a-b469-aa8358526fe6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.104749 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.105471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.105754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.106029 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.106713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.108594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.137563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1116/2229] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:20 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:20.141718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.151519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc170159-f04d-44a1-ace4-f956aa4e96ed admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.190256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:20.208026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.208726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.209007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.209298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.209942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.211853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.233277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1114/2230] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:20 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:20.237504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.248430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2600e504-306c-424b-a4f0-9daac5cf304f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.253298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.253895 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.254174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.254461 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.255103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.256690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.266998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1117/2231] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:20.270924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.280712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db1e2f3-5d6f-4cd5-9815-81b30c741187 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.285303 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.285894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.286170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.286458 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.287108 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.288813 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8c84a-b5d0-4a58-9c05-66622b3ef045 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.299327 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1115/2232] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:20.303464 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.313731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a6b383-e470-4ae7-9fed-0bee7d6ef160 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.318297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.318968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.319249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.319520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] PATH_INFO: `/v3/projects/fbde77953dd749f8bde911c9e96b6482/users/d87b35758d7449a190755c6e6e2d69ad/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.334532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fbde77953dd749f8bde911c9e96b6482, user_id=d87b35758d7449a190755c6e6e2d69ad, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.337219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.350384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1118/2233] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:20 2022] PUT /identity/v3/projects/fbde77953dd749f8bde911c9e96b6482/users/d87b35758d7449a190755c6e6e2d69ad/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:22:20.362044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.362611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.362712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.392047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] MFA Rules not processed for user `d87b35758d7449a190755c6e6e2d69ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:20.393301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.454298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1116/2234] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:20 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:20.465663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.467346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.524616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70705eb-87c9-48be-a7b2-4e431753d1da admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.528445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.529045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.529328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.529604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.537780 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.539249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52686006-85d5-4c12-a8e2-4e44bec6ef64 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.548599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1119/2235] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:20.567846 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.630964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952581c-d9c8-43a7-9c76-58f0cd618a4b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.635509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.636587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.636867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.637141 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] PATH_INFO: `/v3/users/d87b35758d7449a190755c6e6e2d69ad` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.638259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d87b35758d7449a190755c6e6e2d69ad)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.640196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.690204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd87b35758d7449a190755c6e6e2d69ad'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:20.694635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:20.698299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd87b35758d7449a190755c6e6e2d69ad'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:20.699661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd87b35758d7449a190755c6e6e2d69ad'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:20.707118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd87b35758d7449a190755c6e6e2d69ad'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:20.707738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd87b35758d7449a190755c6e6e2d69ad'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:20.724698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1117/2236] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:20 2022] DELETE /identity/v3/users/d87b35758d7449a190755c6e6e2d69ad => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:20.729118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.729422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.730229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.749756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:20.750967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.807302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1120/2237] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:20.817785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:20.819203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:20.876361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231eefc4-005a-43d9-9191-539f74400570 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:20.880290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:20.880838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:20.881156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:20.881394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:20.890416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:20.891892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1be547f-1f46-443a-98f3-d72db0e2b87a service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:20.901216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1118/2238] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:22:21.393003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.394377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:21.408390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d02486-7cc7-4f2e-a0c7-06d0ebab5363 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.413616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.414422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.414734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.415010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] PATH_INFO: `/v3/projects/fbde77953dd749f8bde911c9e96b6482` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.416140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fbde77953dd749f8bde911c9e96b6482)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.418132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.434941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:21.467553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbde77953dd749f8bde911c9e96b6482', 'request_id': 'req-8a801b51-6184-468d-93b4-1937ea1c9777'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:21.473857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbde77953dd749f8bde911c9e96b6482', 'request_id': 'req-8a801b51-6184-468d-93b4-1937ea1c9777'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:21.479606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbde77953dd749f8bde911c9e96b6482', 'request_id': 'req-8a801b51-6184-468d-93b4-1937ea1c9777'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:21.491861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] The token cache is being invalidate because project fbde77953dd749f8bde911c9e96b6482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:21.492152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:21.492770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-8a801b51-6184-468d-93b4-1937ea1c9777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:21.494251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1121/2239] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:21 2022] DELETE /identity/v3/projects/fbde77953dd749f8bde911c9e96b6482 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:21.500346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.500827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.500926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.520845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:21.522045 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:21.580183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1119/2240] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:22:21.589658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.589886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.590255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.610070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:21.611279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:21.619505 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1122/2241] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:21.624340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.634157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b934d0-44c8-4834-a55d-1b48b37207d2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.639484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.640080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.640362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.640638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.641283 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:22:21.641618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.643306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.654789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1120/2242] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:22:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:22:21.716188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.725537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017aa281-007e-4daa-bd14-ae6444a3b740 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.730309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.730937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.731220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.731497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.732130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.734027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.761676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1123/2243] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:21 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:21.765937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.775649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2f1d5-7669-4b55-a0f2-6f14357e2b0a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.832209 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.832805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.833088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.833369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.834004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.835936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.843932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:21.858536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1121/2244] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:21 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:21.862667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.873837 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187fe2a9-8308-4792-a135-d3e3d933b615 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.878834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.879426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.879701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.879972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.880599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.882201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.893455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1124/2245] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:21.897386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.907005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bea7733-7ad5-4575-8ee5-22251c31a988 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.911488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.912073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.912352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.912624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.913252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.914861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3bee13-8e8d-49d1-9c7f-a3acefacbe88 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.926676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1122/2246] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:21.930741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:21.941614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75f417b-cc8d-45cc-9ef2-2c18d7d01b82 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:21.946141 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:21.946804 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.947083 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.947356 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] PATH_INFO: `/v3/projects/05003d9f8b08434abccc1b50a38b5e2d/users/8952a599e05d463bb2128eb17ce9e836/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:21.962364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05003d9f8b08434abccc1b50a38b5e2d, user_id=8952a599e05d463bb2128eb17ce9e836, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:21.965436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:21.977840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1125/2247] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:21 2022] PUT /identity/v3/projects/05003d9f8b08434abccc1b50a38b5e2d/users/8952a599e05d463bb2128eb17ce9e836/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:21.988660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:21.988967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:21.989235 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.016861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] MFA Rules not processed for user `8952a599e05d463bb2128eb17ce9e836`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:22.018054 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:22.079722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1123/2248] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:22.084203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.144930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fbb203-7682-4873-8146-c29df010a128 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.149433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.150028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.150337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.150642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.151262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.153377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.179689 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1126/2249] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:22.184659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.195284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d83240-7fd2-4f14-872a-55064fc78836 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.250290 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.250922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.251201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.251477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.252096 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.254257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.274208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1124/2250] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:22.278259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.288613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24743f00-d625-4fd6-9f51-2e420fe52199 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.293893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.294509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.294789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.295064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.295731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.297334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.309608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1127/2251] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:22.313625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.323611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4ce333-bc72-4c4c-bd69-3e02378ae266 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.328212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.328802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.329080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.329352 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.329986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.331607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bf030-d68c-4cff-b02a-b9ce5d270b41 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.341261 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1125/2252] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:22.345237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.355782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352b00bc-0307-48c2-aff2-615b968954ca admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.360327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.360968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.361251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.361527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] PATH_INFO: `/v3/projects/464847f42eb146a28992b52ef643cf9c/users/dd0e655eb9e0473ca4fa07d982648364/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.374015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:22.377412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] RBAC: Authorizing `identity:create_grant(project_id=464847f42eb146a28992b52ef643cf9c, user_id=dd0e655eb9e0473ca4fa07d982648364, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.380602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.394660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1128/2253] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:22 2022] PUT /identity/v3/projects/464847f42eb146a28992b52ef643cf9c/users/dd0e655eb9e0473ca4fa07d982648364/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:22.405835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.406064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.406339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.434631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] MFA Rules not processed for user `dd0e655eb9e0473ca4fa07d982648364`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:22.435957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:22.468978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:22.501750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1126/2254] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:22.506465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.568096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f154cd2b-1dd6-4a8a-83b9-220a57d74204 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.572840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.573514 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.573784 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.574058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.574713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.576601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.602362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1129/2255] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:22.607187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.617557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4f54f-33b1-414d-bd3a-114aea53f5d6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.673112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.673634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.673915 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.674189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.674849 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.676742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.697698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1127/2256] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/users => generated 299 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:22.701850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.712158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16efd5a-cb3e-4e7e-b231-34b2f7d63869 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.717675 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.718259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.718572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.718845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.719471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.721077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.732540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1130/2257] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:22.736578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.746608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb63e5-ed46-4453-9fb2-7713825e90f7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.751236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.751877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.752161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.752449 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] PATH_INFO: `/v3/projects/f0a0bfede49f47dc82812d909677fdce/users/5e73535efee54bdfbbe54bbbd7e14165/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.767937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a0bfede49f47dc82812d909677fdce, user_id=5e73535efee54bdfbbe54bbbd7e14165, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.770729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.783772 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1128/2258] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:22 2022] PUT /identity/v3/projects/f0a0bfede49f47dc82812d909677fdce/users/5e73535efee54bdfbbe54bbbd7e14165/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:22.787635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.798532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fff2448-e80f-48ab-af31-6b1450c6d39a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.853232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.853828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.854107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.854398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.855139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.856766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982760d8-6c66-4c9c-86ce-0f7c40ba9110 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.866299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1131/2259] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:22 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:22.870523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:22.880928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7c3fe6-32c4-4121-9abe-3ccf6dea4aea admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:22.885549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:22.886740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.887025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.887331 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] PATH_INFO: `/v3/projects/f0a0bfede49f47dc82812d909677fdce/users/5e73535efee54bdfbbe54bbbd7e14165/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.889259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0a0bfede49f47dc82812d909677fdce, user_id=5e73535efee54bdfbbe54bbbd7e14165, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:22.891926 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:22.902459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1129/2260] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:22 2022] PUT /identity/v3/projects/f0a0bfede49f47dc82812d909677fdce/users/5e73535efee54bdfbbe54bbbd7e14165/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:22.913624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:22.913912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:22.914189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:22.941423 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] MFA Rules not processed for user `5e73535efee54bdfbbe54bbbd7e14165`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:22.942578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:22.989072 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:23.010204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1132/2261] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:22 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:23.022198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:23.023986 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:23.080292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88b655c-5fae-45e7-bbc5-a978d86351ce admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:23.083906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:23.084503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:23.084782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:23.085055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:23.126476 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:23.142408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:23.143859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:23.152815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1130/2262] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:24.671345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:24.672879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:24.689160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676c70aa-0f63-43ca-a190-1d883b766f45 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:24.692720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:24.693323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:24.693601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:24.693873 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:24.749527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:24.750922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:24.759929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1133/2263] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:24 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:25.284947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:25.294117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6d945f-f466-4ce2-8728-3ced264237c3 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:25.297740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:25.298335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:25.298655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:25.298931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:25.307720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:25.309785 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65a8822-3879-413b-9fed-402bcd8d0eea service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:25.318980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1131/2264] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:29.211997 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:29.274859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec62932-2f39-45e8-adea-fc04c7dbae0e service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:29.279435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:29.280054 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:29.280332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:29.280599 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] PATH_INFO: `/v3/users/8952a599e05d463bb2128eb17ce9e836` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:29.281684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8952a599e05d463bb2128eb17ce9e836)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:29.283607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:29.329096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8952a599e05d463bb2128eb17ce9e836'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.336000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8952a599e05d463bb2128eb17ce9e836'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.342616 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8952a599e05d463bb2128eb17ce9e836'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.343245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8952a599e05d463bb2128eb17ce9e836'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.343866 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8952a599e05d463bb2128eb17ce9e836'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.361063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1134/2265] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:29 2022] DELETE /identity/v3/users/8952a599e05d463bb2128eb17ce9e836 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:29.365534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:29.366029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:29.366136 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:29.385673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:29.388005 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:29.446535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1132/2266] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:29.456783 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:29.458181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:29.465204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:29.520050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808da6b9-f490-4586-96f6-b5e817d85e21 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:29.523706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:29.524307 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:29.524590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:29.524864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:29.534203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:29.535667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:29.545632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1135/2267] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:29.818667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:29.820092 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:29.832635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe73ab3-a77a-4e16-a822-a065dfbb5859 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:29.837237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:29.837874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:29.838180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:29.838485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] PATH_INFO: `/v3/projects/05003d9f8b08434abccc1b50a38b5e2d` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:29.840241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=05003d9f8b08434abccc1b50a38b5e2d)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:29.842159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:29.888968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05003d9f8b08434abccc1b50a38b5e2d', 'request_id': 'req-3a94d069-4052-4db7-9cb2-3ad17a442f9f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.895436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05003d9f8b08434abccc1b50a38b5e2d', 'request_id': 'req-3a94d069-4052-4db7-9cb2-3ad17a442f9f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.901297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05003d9f8b08434abccc1b50a38b5e2d', 'request_id': 'req-3a94d069-4052-4db7-9cb2-3ad17a442f9f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.914169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] The token cache is being invalidate because project 05003d9f8b08434abccc1b50a38b5e2d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:29.914470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.915093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:29.916590 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1133/2268] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:29 2022] DELETE /identity/v3/projects/05003d9f8b08434abccc1b50a38b5e2d => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:29.921476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:29.932418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4406fc5-332f-41cb-b4f3-5a0100237318 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:29.987021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:29.987657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:29.987936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:29.988213 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] PATH_INFO: `/v3/users/dd0e655eb9e0473ca4fa07d982648364` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:29.989386 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd0e655eb9e0473ca4fa07d982648364)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:29.991311 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:30.037285 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd0e655eb9e0473ca4fa07d982648364'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.044673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd0e655eb9e0473ca4fa07d982648364'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.051256 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd0e655eb9e0473ca4fa07d982648364'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.051970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd0e655eb9e0473ca4fa07d982648364'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.052587 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd0e655eb9e0473ca4fa07d982648364'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.069940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1136/2269] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:29 2022] DELETE /identity/v3/users/dd0e655eb9e0473ca4fa07d982648364 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:30.339061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:30.340635 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:30.353753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a94d069-4052-4db7-9cb2-3ad17a442f9f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:30.402607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:30.413886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:30.414620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:30.415175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:30.415454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] PATH_INFO: `/v3/projects/464847f42eb146a28992b52ef643cf9c` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:30.416578 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=464847f42eb146a28992b52ef643cf9c)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:30.419043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:30.463959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '464847f42eb146a28992b52ef643cf9c', 'request_id': 'req-b8e187fd-24c5-4dca-be71-6f36e6b542ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.469989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '464847f42eb146a28992b52ef643cf9c', 'request_id': 'req-b8e187fd-24c5-4dca-be71-6f36e6b542ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.475570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '464847f42eb146a28992b52ef643cf9c', 'request_id': 'req-b8e187fd-24c5-4dca-be71-6f36e6b542ab'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.487800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] The token cache is being invalidate because project 464847f42eb146a28992b52ef643cf9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:30.488087 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.488702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.490215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1134/2270] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:30 2022] DELETE /identity/v3/projects/464847f42eb146a28992b52ef643cf9c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:30.494283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:30.496427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:30.510396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354a39ea-847b-4adb-a739-7a2e31fd92f2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:30.534945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:30.567523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:30.568145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:30.568426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:30.568698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] PATH_INFO: `/v3/users/5e73535efee54bdfbbe54bbbd7e14165` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:30.569799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e73535efee54bdfbbe54bbbd7e14165)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:30.571737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:30.619279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e73535efee54bdfbbe54bbbd7e14165'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.626158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e73535efee54bdfbbe54bbbd7e14165'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.633457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e73535efee54bdfbbe54bbbd7e14165'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.634108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e73535efee54bdfbbe54bbbd7e14165'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.634741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-65cdf892-a14c-4fbd-af9d-fd5b6735bdff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e73535efee54bdfbbe54bbbd7e14165'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:30.652466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1137/2271] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:30 2022] DELETE /identity/v3/users/5e73535efee54bdfbbe54bbbd7e14165 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:30.927009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:30.928455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:30.942013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e187fd-24c5-4dca-be71-6f36e6b542ab admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:30.999467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.000220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.000507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.000787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] PATH_INFO: `/v3/projects/f0a0bfede49f47dc82812d909677fdce` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.001910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0a0bfede49f47dc82812d909677fdce)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.004104 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.051285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0a0bfede49f47dc82812d909677fdce', 'request_id': 'req-97285877-89df-4572-90d5-b3d85aaf7ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:31.057361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0a0bfede49f47dc82812d909677fdce', 'request_id': 'req-97285877-89df-4572-90d5-b3d85aaf7ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:31.062880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0a0bfede49f47dc82812d909677fdce', 'request_id': 'req-97285877-89df-4572-90d5-b3d85aaf7ac7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:31.075266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] The token cache is being invalidate because project f0a0bfede49f47dc82812d909677fdce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:31.075559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:31.076190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-97285877-89df-4572-90d5-b3d85aaf7ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:31.077661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1135/2272] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:30 2022] DELETE /identity/v3/projects/f0a0bfede49f47dc82812d909677fdce => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:31.082612 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.082925 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.083203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.103657 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:31.104855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:31.163343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1138/2273] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:31.171827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.172118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.172394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.192220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:31.193426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:31.202130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1136/2274] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:31.206249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.216002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fcf6970-b57c-4497-a6c8-a14f8a7da009 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.220582 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.221207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.221490 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.221761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.222397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:22:31.222731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.224323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.237568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1139/2275] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:22:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:22:31.241920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.251835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10df8182-4359-4561-874a-598b1bccd4e6 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.256742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.257339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.257622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.257897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.258557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.260494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.265051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:31.288131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1137/2276] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:31 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:31.292182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.303101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d75401-18a6-46b3-abb3-556256e1d9e7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.332334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:31.360684 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.361283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.361562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.361838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.362494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.364401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.388016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1140/2277] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:31 2022] POST /identity/v3/users => generated 285 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:31.389337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.399779 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ba6c83-8945-41a3-980c-85655ca48c47 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.405340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.405993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.406220 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.406526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.407194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.409703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.419897 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1138/2278] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:31.424317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.434420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b2c5fa8-83fb-47a2-ab19-6e233f4c649a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.438953 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.439548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.439828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.440102 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.440734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.442327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bd67f6-08b3-4fbc-b3d7-be288424a775 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.452820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1141/2279] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:31.456966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.467586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c77471-dec0-48e7-9673-2868c38b09c5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.472174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.472817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.473107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.473387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] PATH_INFO: `/v3/projects/aa248af6555847989489ffa814d3a2d7/users/2dca58713d0940d7a6acbbcc251d1179/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.488362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa248af6555847989489ffa814d3a2d7, user_id=2dca58713d0940d7a6acbbcc251d1179, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.491065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.504681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1139/2280] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:31 2022] PUT /identity/v3/projects/aa248af6555847989489ffa814d3a2d7/users/2dca58713d0940d7a6acbbcc251d1179/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:31.516471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.516972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.517081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.545036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] MFA Rules not processed for user `2dca58713d0940d7a6acbbcc251d1179`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:31.546258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:31.608270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1142/2281] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:31 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:31.619924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:31.621689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:31.700416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca312b09-c622-441c-8e18-bf9c5325055b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:31.704113 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:31.704715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:31.704996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:31.705268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:31.713584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:31.715643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:31.724541 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1140/2282] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:41.604807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=51,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:41.720514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=56,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:46.933832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:46.935314 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:46.991284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eca5bf0-9ac4-4957-9aaa-e9e2482b3dac None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:46.994828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:46.995454 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:46.995726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:46.995994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:46.997660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:47.076330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:47.077765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:47.096198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1143/2283] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:46 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:50.527525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:50.529008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:50.637399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eeaad5-d4ad-409c-9ce2-869d37477e73 service glance] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:50.644216 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:50.644874 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:50.645158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:50.645457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:50.647286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:50.662727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:50.664626 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:50.677741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1141/2284] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:55.312479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:55.314121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:55.376620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98bd4f5c-387c-4409-8137-1ccfb3226065 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:55.381364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:55.381960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:55.382242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:55.382553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:55.384264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:55.424066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:55.446785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:55.448199 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:55.457992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1144/2285] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:22:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:22:58.729078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:58.730664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:58.745022 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc60eac-6d91-430d-95d0-4ee9c8ded712 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:58.749637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:58.750254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:58.750565 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:58.750837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] PATH_INFO: `/v3/users/c9dba6c5f7e9472faa5aa963758d78fe` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:58.752071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9dba6c5f7e9472faa5aa963758d78fe)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:58.754055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:58.800401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9dba6c5f7e9472faa5aa963758d78fe'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:58.807975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9dba6c5f7e9472faa5aa963758d78fe'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:58.808662 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9dba6c5f7e9472faa5aa963758d78fe'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:58.815891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9dba6c5f7e9472faa5aa963758d78fe'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:58.816487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-dc8aa8ad-4dc8-4334-a65d-3117087ec2a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9dba6c5f7e9472faa5aa963758d78fe'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:58.834077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1142/2286] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:22:58 2022] DELETE /identity/v3/users/c9dba6c5f7e9472faa5aa963758d78fe => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:59.010097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.020941 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67648d65-c39e-4b6f-abd9-960c21c27d60 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.080343 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.080968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.081246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.081519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] PATH_INFO: `/v3/projects/fa4d62bf272e4ce69022518ff362813f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.082652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa4d62bf272e4ce69022518ff362813f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.084786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.130080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa4d62bf272e4ce69022518ff362813f', 'request_id': 'req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:59.136477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa4d62bf272e4ce69022518ff362813f', 'request_id': 'req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:59.141916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa4d62bf272e4ce69022518ff362813f', 'request_id': 'req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:59.153845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] The token cache is being invalidate because project fa4d62bf272e4ce69022518ff362813f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:22:59.154131 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:59.154753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-af54ea3c-9b7d-48d3-bac5-d10fbb9f64c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:22:59.156221 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1145/2287] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:22:59 2022] DELETE /identity/v3/projects/fa4d62bf272e4ce69022518ff362813f => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:59.161097 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.161383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.162189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.183932 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:59.185124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:59.234333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:59.245944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1143/2288] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:59.253947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.254159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.254468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.274236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:59.275540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:59.283380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1146/2289] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:59.288120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.297797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6776abc3-f905-4b24-a437-3a9f9e5d9a78 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.302329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.303516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.303828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.304070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.304705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:22:59.305008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.306698 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.317968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1144/2290] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:22:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:22:59.322579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.331825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b337ed-4a32-488a-a014-3e0b00f8814a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.337230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.337826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.338107 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.338381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.339059 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.340945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.368344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1147/2291] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:59.372538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.383066 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a3e2b8-1579-49c7-96a4-275d7e369cd8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.437315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.438170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.438503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.438824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.439442 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.441329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.462274 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1145/2292] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/users => generated 290 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:59.467139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.478501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e77c3c-3f39-4e2c-bc7a-be239bc922fe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.483031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.483623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.483904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.484177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.484807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.486398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.496187 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1148/2293] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:59.500201 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.511189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e31354-77fc-4ca0-88dd-bbc7c77d0402 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.515676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.516270 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.516555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.516828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.517457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.519090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3514faa-0572-4183-8800-3f3e9c2b12af admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.529997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1146/2294] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:59.534249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.544550 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c12fa-e550-429e-9a40-6301c8837aee admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.549122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.549752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.550031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.550302 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] PATH_INFO: `/v3/projects/31c8748c800d4a319cab168ad779dc87/users/9de7597349af44a1b83a1fb912b24544/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.565540 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31c8748c800d4a319cab168ad779dc87, user_id=9de7597349af44a1b83a1fb912b24544, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.568754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.592779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1149/2295] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:59 2022] PUT /identity/v3/projects/31c8748c800d4a319cab168ad779dc87/users/9de7597349af44a1b83a1fb912b24544/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:59.594818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.595172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.595579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.626032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] MFA Rules not processed for user `9de7597349af44a1b83a1fb912b24544`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:22:59.627508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:22:59.695190 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1147/2296] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:22:59.699628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.704138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:59.765848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d63dc-6734-453e-a549-06bd67c9b278 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.770416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.771052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.771547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.771881 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.772528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.774480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.800992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1150/2297] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:59.806276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.816862 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46ee589-32c8-4c85-a617-882a11766dec None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.864797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:22:59.875262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.875861 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.876138 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.876409 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.877064 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.878988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.900094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1148/2298] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:22:59 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:22:59.904040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.914513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17df8a7c-39af-4b74-aec9-196886fd819b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.919325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.919912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.920191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.920462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.921114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.922743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.934272 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1151/2299] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:22:59.940422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:22:59.952789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e08df6-b20e-43b2-8d59-1c7984a8331b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:22:59.957232 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:22:59.957872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:22:59.958148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:22:59.958421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] PATH_INFO: `/v3/projects/90183e001fee4733945fa407d4c3be11/users/921df7b9cfe74341bbe763aca19eeaf7/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:22:59.973942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=90183e001fee4733945fa407d4c3be11, user_id=921df7b9cfe74341bbe763aca19eeaf7, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:22:59.977074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:22:59.989255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1149/2300] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:22:59 2022] PUT /identity/v3/projects/90183e001fee4733945fa407d4c3be11/users/921df7b9cfe74341bbe763aca19eeaf7/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:22:59.993483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:00.003586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb4cccf-6b0c-4c1f-88fb-e1d99872dcfe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:00.061631 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:00.062282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:00.062400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:00.062709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:00.063352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:00.064939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8720fa9c-fde8-4c85-9b28-edf015a8631b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:00.075347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1152/2301] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:00.079514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:00.090163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67163d3-5c9f-4cba-b627-cb29e9370d1b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:00.095350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:00.096027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:00.096311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:00.096587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] PATH_INFO: `/v3/projects/90183e001fee4733945fa407d4c3be11/users/921df7b9cfe74341bbe763aca19eeaf7/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:00.098557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90183e001fee4733945fa407d4c3be11, user_id=921df7b9cfe74341bbe763aca19eeaf7, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:00.101375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:00.111544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1150/2302] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:00 2022] PUT /identity/v3/projects/90183e001fee4733945fa407d4c3be11/users/921df7b9cfe74341bbe763aca19eeaf7/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:00.122190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:00.122520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:00.122803 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:00.151618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] MFA Rules not processed for user `921df7b9cfe74341bbe763aca19eeaf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:00.152835 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:00.216518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:00.219990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1153/2303] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:00 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:00.231436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:00.233285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:00.290532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee126dc6-b6a6-4479-adef-997977a069b1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:00.294162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:00.294794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:00.295079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:00.295357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:00.305144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:00.306594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:00.315835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1151/2304] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:00 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:00.391629 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:00.401317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3935b3df-a893-4d20-90c3-0857dc9acbf6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:00.404893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:00.405479 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:00.405759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:00.406033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:00.462988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:00.464455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9351bb-7da7-454c-bfd6-02e0def79367 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:00.473192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1154/2305] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:10.311783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=46,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:10.469198 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=29 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:16.758098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:16.819899 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23963ceb-54e0-4e4f-8cb7-a4ead20add15 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:16.824436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:16.825058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:16.825342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:16.825620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] PATH_INFO: `/v3/users/9de7597349af44a1b83a1fb912b24544` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:16.827545 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de7597349af44a1b83a1fb912b24544)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:16.829450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:16.874803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de7597349af44a1b83a1fb912b24544'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:16.882664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de7597349af44a1b83a1fb912b24544'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:16.883321 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de7597349af44a1b83a1fb912b24544'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:16.891048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de7597349af44a1b83a1fb912b24544'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:16.891854 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de7597349af44a1b83a1fb912b24544'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:16.909450 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1152/2306] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:16 2022] DELETE /identity/v3/users/9de7597349af44a1b83a1fb912b24544 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:16.919082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:16.920219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:16.920327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:16.939201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:16.940394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:16.999876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1155/2307] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:17.009551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:17.011004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:17.072732 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed72ffd-71e8-4e9b-97ea-7af1f7b324d7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:17.077734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:17.078312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:17.078637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:17.078898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:17.088168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:17.089660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:17.099806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1153/2308] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:17.378077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:17.379536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:17.392070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f0ddc0-5c14-48ff-a25c-c9893a6ee994 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:17.396799 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:17.397418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:17.397700 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:17.397980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] PATH_INFO: `/v3/projects/31c8748c800d4a319cab168ad779dc87` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:17.399121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31c8748c800d4a319cab168ad779dc87)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:17.401011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:17.456879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31c8748c800d4a319cab168ad779dc87', 'request_id': 'req-ef47e80f-e6b8-4081-ab39-070eadf54bc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.463108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31c8748c800d4a319cab168ad779dc87', 'request_id': 'req-ef47e80f-e6b8-4081-ab39-070eadf54bc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.468834 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31c8748c800d4a319cab168ad779dc87', 'request_id': 'req-ef47e80f-e6b8-4081-ab39-070eadf54bc5'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.480701 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] The token cache is being invalidate because project 31c8748c800d4a319cab168ad779dc87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:17.480992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.481611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.483250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1156/2309] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:17 2022] DELETE /identity/v3/projects/31c8748c800d4a319cab168ad779dc87 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:17.487288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:17.496679 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606cbb1-7353-4de5-8688-755d861ff7e4 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:17.516561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:17.557631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:17.558252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:17.558561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:17.558833 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] PATH_INFO: `/v3/users/921df7b9cfe74341bbe763aca19eeaf7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:17.559952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=921df7b9cfe74341bbe763aca19eeaf7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:17.562936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:17.607725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '921df7b9cfe74341bbe763aca19eeaf7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.614501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '921df7b9cfe74341bbe763aca19eeaf7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.615150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '921df7b9cfe74341bbe763aca19eeaf7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.622048 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '921df7b9cfe74341bbe763aca19eeaf7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.622670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ee0ddad4-9234-463d-8598-6929dc6a4e91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '921df7b9cfe74341bbe763aca19eeaf7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:17.641171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1154/2310] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:17 2022] DELETE /identity/v3/users/921df7b9cfe74341bbe763aca19eeaf7 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:17.909320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:17.910867 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:17.925236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef47e80f-e6b8-4081-ab39-070eadf54bc5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:17.981779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:17.982986 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:17.983292 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:17.983570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] PATH_INFO: `/v3/projects/90183e001fee4733945fa407d4c3be11` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:17.984671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] RBAC: Authorizing `identity:delete_project(project_id=90183e001fee4733945fa407d4c3be11)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:17.986593 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.016066 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:18.036227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90183e001fee4733945fa407d4c3be11', 'request_id': 'req-539694f8-8a8a-427d-98b7-5f54d0f1f86c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:18.043549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90183e001fee4733945fa407d4c3be11', 'request_id': 'req-539694f8-8a8a-427d-98b7-5f54d0f1f86c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:18.050034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90183e001fee4733945fa407d4c3be11', 'request_id': 'req-539694f8-8a8a-427d-98b7-5f54d0f1f86c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:18.062520 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] The token cache is being invalidate because project 90183e001fee4733945fa407d4c3be11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:18.062807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:18.063434 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-539694f8-8a8a-427d-98b7-5f54d0f1f86c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:18.064906 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1157/2311] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:17 2022] DELETE /identity/v3/projects/90183e001fee4733945fa407d4c3be11 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:18.069717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.070186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.070285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.089223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:18.090401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:18.148628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1155/2312] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:18.158738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.158960 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.159250 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.179830 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:18.181027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:18.189127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1158/2313] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:18.193179 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.203086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baeaa17-d95a-4cad-9b6c-50d01e2177d3 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.207660 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.208268 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.208546 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.208822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.209461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:23:18.209765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.211390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.223564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1156/2314] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:23:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:23:18.227962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.238501 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7c2d53-c61e-4aa5-864d-9d32e564ae6a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.243063 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.243673 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.243933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.244201 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.244818 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.246712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.278176 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1159/2315] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:18.280073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.296294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82df7b6d-9d0f-40ca-a9ca-8060d3d08790 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.319724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:18.354160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.354853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.355129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.355412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.356026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.359023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.383617 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1157/2316] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/users => generated 283 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:18.387786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.398669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ae086e-08cf-4083-859d-1905ceb06501 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.403179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.403786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.404045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.404313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.405937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.407562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.417241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1160/2317] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:23:18.421673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.431564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6fc5e6-6fcb-4724-bf16-bac4711dcb7c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.436249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.436844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.437126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.437402 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.438036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.439667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf96ea1-9808-4ced-8fbd-cedc9897ef1d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.450149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1158/2318] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:18.455242 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.465097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62342ae4-0183-4d1d-86c6-e520ef25d2ab admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.470563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.471203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.471483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.471756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] PATH_INFO: `/v3/projects/46efdca67fbe41fabf3d53aaa9c54525/users/2b7cdb57a5be494992bce6fc25935f4e/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.485917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46efdca67fbe41fabf3d53aaa9c54525, user_id=2b7cdb57a5be494992bce6fc25935f4e, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.488623 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.502333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1161/2319] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:18 2022] PUT /identity/v3/projects/46efdca67fbe41fabf3d53aaa9c54525/users/2b7cdb57a5be494992bce6fc25935f4e/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:18.513486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.514308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.514594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.542176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] MFA Rules not processed for user `2b7cdb57a5be494992bce6fc25935f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:18.543420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:18.609167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1159/2320] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:18.614517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.676304 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a397eb2-3c8c-48d8-a03d-20e8bf4e9ed1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.681127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.681742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.682024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.682299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.682985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.685065 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.715230 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1162/2321] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/projects => generated 361 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:23:18.716469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.727314 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e72b5a-e3ec-41c1-b7aa-e69a32dbef7b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.784137 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.784743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.785051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.785323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.785965 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.787923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.812157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1160/2322] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:18 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:18.813360 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.823788 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa95f4af-7d7f-4436-bb4b-e0aca6934e48 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.828590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.829188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.829471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.829748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.830380 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.832040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.843241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1163/2323] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:18.847205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.857481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef599f32-1526-4a58-b063-e4ced9903390 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.863805 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.864471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.864754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.865029 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] PATH_INFO: `/v3/projects/2439cffa952c4c048046fcae78acfcd4/users/d3c3f2016c57426cbdd6c1b61d5e46e3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.881157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2439cffa952c4c048046fcae78acfcd4, user_id=d3c3f2016c57426cbdd6c1b61d5e46e3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.883870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.898287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1161/2324] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:18 2022] PUT /identity/v3/projects/2439cffa952c4c048046fcae78acfcd4/users/d3c3f2016c57426cbdd6c1b61d5e46e3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:18.902215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.913373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c772ad-5197-4a5e-a873-08949ab4f784 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:18.934691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:18.976087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:18.976697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:18.976978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:18.977254 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:18.977915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:18.979583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710af343-9a03-4a9a-8d2e-487a6fd671d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:18.990025 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1164/2325] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:18.994213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:18.999867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:19.006074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e68994-a3a1-40ed-bd59-caf454302256 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:19.010996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:19.011641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.011923 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.012200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] PATH_INFO: `/v3/projects/2439cffa952c4c048046fcae78acfcd4/users/d3c3f2016c57426cbdd6c1b61d5e46e3/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.014607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2439cffa952c4c048046fcae78acfcd4, user_id=d3c3f2016c57426cbdd6c1b61d5e46e3, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:19.017134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:19.027162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1162/2326] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:18 2022] PUT /identity/v3/projects/2439cffa952c4c048046fcae78acfcd4/users/d3c3f2016c57426cbdd6c1b61d5e46e3/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:19.037628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.037916 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.038193 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.065738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] MFA Rules not processed for user `d3c3f2016c57426cbdd6c1b61d5e46e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:19.066970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:19.133299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1165/2327] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:19.144972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:19.146691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:19.206702 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadcdf16-b483-4177-aca7-dbba79775e32 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:19.210198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:19.210841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.211118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.211388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.220914 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:19.222333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806772a5-4b9b-4010-85a8-d5bed7fe94bb service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:19.231611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1163/2328] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:19 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:19.661880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:19.725383 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83547acd-7f92-4faa-ad30-dd95faa2f0e4 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:19.729782 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:19.730407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.730730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.731006 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] PATH_INFO: `/v3/users/2b7cdb57a5be494992bce6fc25935f4e` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.732114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b7cdb57a5be494992bce6fc25935f4e)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:19.734090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:19.780548 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b7cdb57a5be494992bce6fc25935f4e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:19.781924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:19.789271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b7cdb57a5be494992bce6fc25935f4e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:19.797367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b7cdb57a5be494992bce6fc25935f4e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:19.798027 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b7cdb57a5be494992bce6fc25935f4e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:19.798680 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b7cdb57a5be494992bce6fc25935f4e'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:19.816405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1166/2329] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:19 2022] DELETE /identity/v3/users/2b7cdb57a5be494992bce6fc25935f4e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:19.820853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.821156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.821432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.840275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:19.841536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:19.901941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1164/2330] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:19.912494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:19.913884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:19.980766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834364c-80e0-4f8d-afce-492ea2d7df56 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:19.984488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:19.985089 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:19.985539 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:19.985822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:19.995076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:19.996495 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:20.007846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1167/2331] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:20.275526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:20.276961 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:20.289788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b71f5-930e-45fa-9201-93eacd800c82 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:20.294296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:20.294968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:20.295251 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:20.295529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] PATH_INFO: `/v3/projects/46efdca67fbe41fabf3d53aaa9c54525` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:20.296715 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46efdca67fbe41fabf3d53aaa9c54525)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:20.298646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:20.345706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46efdca67fbe41fabf3d53aaa9c54525', 'request_id': 'req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.352148 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46efdca67fbe41fabf3d53aaa9c54525', 'request_id': 'req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.357957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46efdca67fbe41fabf3d53aaa9c54525', 'request_id': 'req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.370198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] The token cache is being invalidate because project 46efdca67fbe41fabf3d53aaa9c54525 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:20.370507 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.371121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.372599 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1165/2332] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:20 2022] DELETE /identity/v3/projects/46efdca67fbe41fabf3d53aaa9c54525 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:20.377231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:20.386562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584e403-facb-4ada-adfa-24d01fc73f78 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:20.442518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:20.443165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:20.443445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:20.443718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] PATH_INFO: `/v3/users/d3c3f2016c57426cbdd6c1b61d5e46e3` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:20.444824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3c3f2016c57426cbdd6c1b61d5e46e3)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:20.446759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:20.491712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3c3f2016c57426cbdd6c1b61d5e46e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.498416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3c3f2016c57426cbdd6c1b61d5e46e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.506268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3c3f2016c57426cbdd6c1b61d5e46e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.506897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3c3f2016c57426cbdd6c1b61d5e46e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.507494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-1df41dbf-271c-4825-bc8f-42e8ffc68d35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3c3f2016c57426cbdd6c1b61d5e46e3'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.512588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:20.525779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1168/2333] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:20 2022] DELETE /identity/v3/users/d3c3f2016c57426cbdd6c1b61d5e46e3 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:20.804145 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:20.805646 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:20.819400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8e2d3e-1250-49ed-99b8-d3bc1436a221 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:20.831803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:20.877755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:20.878378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:20.878707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:20.878978 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] PATH_INFO: `/v3/projects/2439cffa952c4c048046fcae78acfcd4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:20.880071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] RBAC: Authorizing `identity:delete_project(project_id=2439cffa952c4c048046fcae78acfcd4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:20.881937 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:20.929489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2439cffa952c4c048046fcae78acfcd4', 'request_id': 'req-5178abd4-0ce5-4230-ba61-e211856251da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.935605 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2439cffa952c4c048046fcae78acfcd4', 'request_id': 'req-5178abd4-0ce5-4230-ba61-e211856251da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.941305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2439cffa952c4c048046fcae78acfcd4', 'request_id': 'req-5178abd4-0ce5-4230-ba61-e211856251da'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.955119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] The token cache is being invalidate because project 2439cffa952c4c048046fcae78acfcd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:20.955407 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.956021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5178abd4-0ce5-4230-ba61-e211856251da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:20.957477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1166/2334] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:20 2022] DELETE /identity/v3/projects/2439cffa952c4c048046fcae78acfcd4 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:20.963535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:20.963817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:20.964096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:20.983967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:20.985160 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.046487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1169/2335] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:21.054634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.054886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.055193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.075998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:21.078044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.087165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1167/2336] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:21.091363 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.102274 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b56384-b8ae-445c-8ee1-b27840167b87 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.108139 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.108984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.109258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.109538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.110317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:23:21.110654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.112284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.123636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1170/2337] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:23:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:23:21.128807 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.138211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0688f740-7b30-4e64-86ba-506c3640c894 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.143996 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.144602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.144885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.145271 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.146132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.148843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.176648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1168/2338] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:21 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:23:21.180448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.191247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c4f9a4-1136-4d55-85cb-3dcf0be3b56b admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.246766 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.247370 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.247652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.247928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.248562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.251038 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.272977 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1171/2339] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:21 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:21.277167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.288519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68dde-43ce-454c-995c-35964746d957 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.293184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.293777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.294059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.294357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.295074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.296820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.306759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1169/2340] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:21.310400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.320660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4f1b13-7279-4773-a61a-6ad0466d4583 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.325245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.325822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.326099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.326373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.327026 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.329377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec9c9c8-1193-43b0-9405-45f9b617c9a9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.339731 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1172/2341] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:21.343962 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.354257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6bd9a4-f1fe-4d29-8f70-793477ac9792 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.359673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.360382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.360671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.360950 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] PATH_INFO: `/v3/projects/61399dad0bcb443e82d068135ca8eef4/users/d3ca1aba63854c2884cdc54736f2ae34/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.375885 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61399dad0bcb443e82d068135ca8eef4, user_id=d3ca1aba63854c2884cdc54736f2ae34, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.378563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.392302 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1170/2342] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:21 2022] PUT /identity/v3/projects/61399dad0bcb443e82d068135ca8eef4/users/d3ca1aba63854c2884cdc54736f2ae34/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:23:21.402553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.402843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.403149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.431346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] MFA Rules not processed for user `d3ca1aba63854c2884cdc54736f2ae34`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:21.432536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.472919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:21.497018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1173/2343] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:21.509474 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.511355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.556065 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:21.569518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c86d939-3b79-4365-93b1-43d4a0aabfc1 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.573098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19022fccab844d1dac60f5eddf67f323', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.573694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.573976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.574250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.582575 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.583988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189ae04f-e8f6-4bf3-bcbf-40ac5b0f783b service glance] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.592529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1171/2344] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:21.699871 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.701273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.768097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7073b39-8f3d-426b-a631-dc641d704b0c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:21.772741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:21.773390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.773695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.773963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] PATH_INFO: `/v3/users/2dca58713d0940d7a6acbbcc251d1179` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.775576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2dca58713d0940d7a6acbbcc251d1179)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:21.777819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:21.828864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2dca58713d0940d7a6acbbcc251d1179'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:21.835647 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2dca58713d0940d7a6acbbcc251d1179'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:21.844416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2dca58713d0940d7a6acbbcc251d1179'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:21.845075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2dca58713d0940d7a6acbbcc251d1179'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:21.845807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2dca58713d0940d7a6acbbcc251d1179'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:21.868225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1174/2345] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:21 2022] DELETE /identity/v3/users/2dca58713d0940d7a6acbbcc251d1179 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:21.874944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:21.875254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:21.875627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:21.896539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:21.897823 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:21.961938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1172/2346] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:23:21.973052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:21.974796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:22.050654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0602c18e-cb57-4c7b-b707-0ac1089da207 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.050654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.050654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.050654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.051815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.059411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.060840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c780a1fe-2210-47d4-8b08-b0b72fa87ced service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.071999 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1175/2347] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:23:22.350798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.352382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:22.366718 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66518c16-72d9-461e-b5af-164915dfda6c None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.371786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.372420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.372700 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.372972 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] PATH_INFO: `/v3/projects/aa248af6555847989489ffa814d3a2d7` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.374130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa248af6555847989489ffa814d3a2d7)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.376158 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.427237 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa248af6555847989489ffa814d3a2d7', 'request_id': 'req-26baa187-d6f3-4ad2-8247-87687a2df70e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:22.433585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa248af6555847989489ffa814d3a2d7', 'request_id': 'req-26baa187-d6f3-4ad2-8247-87687a2df70e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:22.440051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa248af6555847989489ffa814d3a2d7', 'request_id': 'req-26baa187-d6f3-4ad2-8247-87687a2df70e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:22.455074 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] The token cache is being invalidate because project aa248af6555847989489ffa814d3a2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:22.455362 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:22.455993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-26baa187-d6f3-4ad2-8247-87687a2df70e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:22.457487 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1173/2348] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:22 2022] DELETE /identity/v3/projects/aa248af6555847989489ffa814d3a2d7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:22.462529 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.462860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.463143 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.472405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:22.485190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:22.486481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:22.551084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1176/2349] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:22.558866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.559155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.559438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.580869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:22.582279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:22.590315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1174/2350] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:22.594768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.604498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7a2858-24a0-4fb4-9afe-f3187b1f8884 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.609067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.609659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.609938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.610212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.610921 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:23:22.611226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.612815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.624135 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1177/2351] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:23:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:23:22.629339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.639619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdd6c69-c54b-42b6-8c5a-796ddaf8d254 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.645259 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.645881 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.646168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.646548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.647208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.649143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.676436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1175/2352] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:22 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:22.680943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.695476 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ff02f-61b9-4d41-b2e3-65c09d86a8d5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.752133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.752736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.753016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.753291 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.753918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.755855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.781121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1178/2353] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:22 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:23:22.785346 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.795636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc667cc-54db-46c3-8a28-a6e0a8ddc326 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.800454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.801053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.801337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.801613 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.802255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.803902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.815869 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1176/2354] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:22.819974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.830738 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af92442d-bfd1-4d3c-a0b0-b3e4ca7c76b1 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.835372 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.835966 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.836248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.836525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.837163 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.838821 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd12568-32d4-43c1-8649-e7b80ccac111 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.849366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1179/2355] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:22.853428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:22.864093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56b74b-91fb-4295-8f0c-5eae18012a45 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:22.870781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:22.872421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.872766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.873042 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] PATH_INFO: `/v3/projects/432641cf60c943ac810f1c55defbf4e0/users/52a62b82d2514ec785e69c61b069bff8/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.889269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=432641cf60c943ac810f1c55defbf4e0, user_id=52a62b82d2514ec785e69c61b069bff8, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:22.892840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:22.905292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1177/2356] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:22 2022] PUT /identity/v3/projects/432641cf60c943ac810f1c55defbf4e0/users/52a62b82d2514ec785e69c61b069bff8/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:22.917389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397b87-bf4b-4fdc-acad-f2f1236fe005 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:22.917691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397b87-bf4b-4fdc-acad-f2f1236fe005 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:22.917955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed397b87-bf4b-4fdc-acad-f2f1236fe005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:22.947555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ed397b87-bf4b-4fdc-acad-f2f1236fe005 None None] MFA Rules not processed for user `52a62b82d2514ec785e69c61b069bff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:22.948757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ed397b87-bf4b-4fdc-acad-f2f1236fe005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:23.009707 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:23.015010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1180/2357] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:23.025631 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:23.027344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:23.033817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:23.090003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26ffae-587a-4c07-95ec-79d9ab823981 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:23.095124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:23.095801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:23.096149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:23.096486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:23.105370 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:23.107107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:23.118388 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1178/2358] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:24.498517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:24.499051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:24.499051 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:24.520759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] MFA Rules not processed for user `52a62b82d2514ec785e69c61b069bff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:24.522100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:24.530271 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1181/2359] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:25.006485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:25.008685 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:25.084413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c27405-21e8-42c3-8fed-498e84663b29 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:25.089632 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:25.090253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:25.090577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:25.090852 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:25.092580 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:25.104457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:25.106011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:25.116253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1179/2360] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:34.003606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:34.006100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:34.090912 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e5ceba-22a5-4009-bc89-b3153d60b4d3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:34.096331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:34.096930 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:34.097211 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:34.097487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:34.099281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:34.111075 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:34.112635 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60819f85-52ca-4849-9c89-dca09bd2b77f service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:34.123549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1182/2361] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:35.111952 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=56 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:37.234934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:37.247776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4cf61b-dd6d-4c94-867c-5b442735e078 service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:37.253248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:37.253843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:37.254124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:37.254399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:37.256156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:37.335498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:37.337689 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:37.356699 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1180/2362] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:42.389585 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79a9f-dae3-4f67-b586-e46e7f12ca9c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:42.390058 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79a9f-dae3-4f67-b586-e46e7f12ca9c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:42.390159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae79a9f-dae3-4f67-b586-e46e7f12ca9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:42.438773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-cae79a9f-dae3-4f67-b586-e46e7f12ca9c None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:42.439964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-cae79a9f-dae3-4f67-b586-e46e7f12ca9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:42.503913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1183/2363] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:23:42 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:42.550317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:42.551766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:42.639927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8082342d-f709-4075-8a82-32ff749fe4d5 service nova] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:42.644879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:42.645471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:42.645747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:42.646016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:42.654143 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:42.655724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:42.663445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1181/2364] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:23:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:23:43.165417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51e35-2734-4c9c-bb40-29c9a03f6f33 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.165706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51e35-2734-4c9c-bb40-29c9a03f6f33 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.165984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb51e35-2734-4c9c-bb40-29c9a03f6f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.186366 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-acb51e35-2734-4c9c-bb40-29c9a03f6f33 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:43.187568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-acb51e35-2734-4c9c-bb40-29c9a03f6f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:43.194649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1184/2365] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:43.207057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:43.218213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7de53e3-531c-421f-bdbb-c72cd85624af service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:43.223193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:43.223789 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.224068 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.224344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.232935 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:43.234535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:43.242554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1182/2366] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:23:43.473191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3fbc-c01d-4a80-bc1a-fb2c7aed155b None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.473659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3fbc-c01d-4a80-bc1a-fb2c7aed155b None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.473765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dab3fbc-c01d-4a80-bc1a-fb2c7aed155b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.493546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2dab3fbc-c01d-4a80-bc1a-fb2c7aed155b None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:43.494762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2dab3fbc-c01d-4a80-bc1a-fb2c7aed155b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:43.502118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1185/2367] 10.222.0.23 () {58 vars in 1009 bytes} [Sun Oct 23 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:43.515768 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:43.526399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b60a9-7f38-42b6-a170-e36e179af615 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:43.532047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:43.532638 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.532919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.533197 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.541439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:43.543116 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:43.551099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1183/2368] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:23:43.874524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.874810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.875086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.894826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:43.895991 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:43.904021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1186/2369] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:23:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:43.915951 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:43.926399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f30640-8226-410b-9035-0dacf68271f0 service swift] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:43.931383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:43.931968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:43.932249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:43.932524 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:43.941550 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:43.943162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2538f8-1394-42eb-9d79-f40cd9e72364 service swift] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:43.947917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:43.953521 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1184/2370] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:23:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Oct 23 02:23:46.043280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:46.076697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:46.108685 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa580ab-6692-4f0b-a71e-37a23e878304 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:46.113166 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:46.113827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:46.114105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:46.114387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] PATH_INFO: `/v3/users/52a62b82d2514ec785e69c61b069bff8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:46.115557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52a62b82d2514ec785e69c61b069bff8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:46.117474 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:46.166224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52a62b82d2514ec785e69c61b069bff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.172862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52a62b82d2514ec785e69c61b069bff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.180518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52a62b82d2514ec785e69c61b069bff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.181219 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52a62b82d2514ec785e69c61b069bff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.181843 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52a62b82d2514ec785e69c61b069bff8'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.201364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1187/2371] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:46 2022] DELETE /identity/v3/users/52a62b82d2514ec785e69c61b069bff8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:46.205824 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:46.206304 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:46.206412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:46.224990 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:46.226195 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:46.284299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1185/2372] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:46.294203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:46.295658 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:46.352353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d9e4d8-2ccd-4cbf-afde-113a1234ee84 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:46.356248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:46.356711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:46.356992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:46.357263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:46.366352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:46.367811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:46.377665 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1188/2373] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:46.544028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:46.545568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:46.563309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7830dda9-4e90-488f-b9c1-e158e30f56ea None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:46.568105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:46.568920 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:46.569333 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:46.569622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] PATH_INFO: `/v3/projects/432641cf60c943ac810f1c55defbf4e0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:46.570973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=432641cf60c943ac810f1c55defbf4e0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:46.572930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:46.624473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '432641cf60c943ac810f1c55defbf4e0', 'request_id': 'req-f6e93482-3375-4fc4-ab5b-64279f0b31e2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.630640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '432641cf60c943ac810f1c55defbf4e0', 'request_id': 'req-f6e93482-3375-4fc4-ab5b-64279f0b31e2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.636095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '432641cf60c943ac810f1c55defbf4e0', 'request_id': 'req-f6e93482-3375-4fc4-ab5b-64279f0b31e2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.650217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] The token cache is being invalidate because project 432641cf60c943ac810f1c55defbf4e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:46.650537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.651153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-f6e93482-3375-4fc4-ab5b-64279f0b31e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:46.652654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1186/2374] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:46 2022] DELETE /identity/v3/projects/432641cf60c943ac810f1c55defbf4e0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:50.858831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:50.860245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:50.872936 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0aae60-f2ae-4a86-9aac-de3a809d681a service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:50.919775 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:50.930124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:50.930774 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:50.931057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:50.931332 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] PATH_INFO: `/v3/users/d3ca1aba63854c2884cdc54736f2ae34` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:50.932447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ca1aba63854c2884cdc54736f2ae34)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:50.934385 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:50.980007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ca1aba63854c2884cdc54736f2ae34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:50.987387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ca1aba63854c2884cdc54736f2ae34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:50.994533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ca1aba63854c2884cdc54736f2ae34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:50.995145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ca1aba63854c2884cdc54736f2ae34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:50.995743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ca1aba63854c2884cdc54736f2ae34'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.013399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1189/2375] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:50 2022] DELETE /identity/v3/users/d3ca1aba63854c2884cdc54736f2ae34 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:23:51.018382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a2c2-4862-4673-88df-18588505cced None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.018888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a2c2-4862-4673-88df-18588505cced None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.019014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437a2c2-4862-4673-88df-18588505cced None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.038892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-5437a2c2-4862-4673-88df-18588505cced None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:51.040086 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5437a2c2-4862-4673-88df-18588505cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:51.098540 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1187/2376] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:51.108817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.110202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:51.167810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3774ef0-217c-47e9-85dc-f35857ce1fa3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.172576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.173172 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.173453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.173722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.182609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.184028 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde3babb-d5a8-4dd6-863d-08d559c36bc7 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.194374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1190/2377] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:51.462834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437a2c2-4862-4673-88df-18588505cced None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.464253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5437a2c2-4862-4673-88df-18588505cced None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:51.476585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437a2c2-4862-4673-88df-18588505cced None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.481526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.482169 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.482472 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.483389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] PATH_INFO: `/v3/projects/61399dad0bcb443e82d068135ca8eef4` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.484688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61399dad0bcb443e82d068135ca8eef4)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.486649 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.509927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:51.538276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61399dad0bcb443e82d068135ca8eef4', 'request_id': 'req-3045b6ad-566a-4b23-89c8-fe396c27e740'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.544608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61399dad0bcb443e82d068135ca8eef4', 'request_id': 'req-3045b6ad-566a-4b23-89c8-fe396c27e740'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.550892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61399dad0bcb443e82d068135ca8eef4', 'request_id': 'req-3045b6ad-566a-4b23-89c8-fe396c27e740'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.563592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] The token cache is being invalidate because project 61399dad0bcb443e82d068135ca8eef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:23:51.563883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.564520 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-3045b6ad-566a-4b23-89c8-fe396c27e740 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:51.566026 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1188/2378] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:23:51 2022] DELETE /identity/v3/projects/61399dad0bcb443e82d068135ca8eef4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:23:51.570838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.571134 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.571412 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.591082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:51.592798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:51.650649 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1191/2379] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:51.659917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.660203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.660478 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.682063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:51.683355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:51.691791 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1189/2380] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:51.695972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.706149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c7e72-99e7-4bf1-822c-41a5b538221d None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.710681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.711276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.711556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.711828 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.712466 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:23:51.712771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.714378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.726280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1192/2381] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:23:51.730418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.739555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a37719c-a1a4-43f7-86aa-a340873ae09f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.744111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.744704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.744989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.745264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.745891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.747835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.775162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1190/2382] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:51 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:51.779405 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.785310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:51.792154 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0a2536-f34c-4fee-bf67-f202d61300cd admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.849281 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.849893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.850174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.850533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.851114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.853047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.878303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1193/2383] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:51 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:51.879422 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.890963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4dce0ca-c354-4f00-9810-035b0600ad1a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.895607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.896208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.896498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.896776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.897421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.899059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.910426 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1191/2384] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:23:51.914328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.924335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb286f-bf2f-4c46-b1ce-a9e34283b09a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.929122 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.929712 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.929994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.930268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.930942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.932544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20b03e6-5d71-4b05-80c6-b90f97dd85d9 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.942998 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1194/2385] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:51 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:51.946868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:51.957031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9be463-ef7d-447a-9821-cecd39ba142a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:51.961593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:51.962226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:51.962548 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:51.962826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] PATH_INFO: `/v3/projects/daa30a51af264eddaef749248da416f5/users/6a6b27aa868a4e32ae7c5f9824854048/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:51.977879 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daa30a51af264eddaef749248da416f5, user_id=6a6b27aa868a4e32ae7c5f9824854048, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:51.980588 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:51.993930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1192/2386] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:51 2022] PUT /identity/v3/projects/daa30a51af264eddaef749248da416f5/users/6a6b27aa868a4e32ae7c5f9824854048/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:52.004264 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.004574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.004849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.034236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] MFA Rules not processed for user `6a6b27aa868a4e32ae7c5f9824854048`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:52.035465 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:52.098975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1195/2387] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:52.103493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.165039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6a40c-7a2d-40e8-ab2e-2a82d0900855 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.169661 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.170267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.170610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.170882 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.171515 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.173513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.200845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1193/2388] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:52.206013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.216573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5420e84-537f-435a-bc02-486e845a3bfe None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.273781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.274384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.274688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.275215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.275826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.277715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.294124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:52.299681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1196/2389] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:52.303781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.318060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13799dc8-1bed-45a5-955f-686727041957 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.323129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.323740 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.324020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.324294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.325000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.328690 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.343123 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1194/2390] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:52.347340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.357493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688b2cd-e465-486a-bb33-18ec0d2bf759 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.362427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.363220 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.363498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.363773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.364407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.366013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550daed-d774-4478-835d-1f7c20a349a8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.380967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1197/2391] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:52.385339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.396159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91899ba8-0d6c-40e5-9b85-1279716cb820 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.401508 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.402394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.402982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.403497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] PATH_INFO: `/v3/projects/df8b4755c14f4da5b3828e1d4e50e2d4/users/f64874a709b74d0387e60026ac4387d6/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.419716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df8b4755c14f4da5b3828e1d4e50e2d4, user_id=f64874a709b74d0387e60026ac4387d6, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.422929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.425186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:52.438488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1195/2392] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:52 2022] PUT /identity/v3/projects/df8b4755c14f4da5b3828e1d4e50e2d4/users/f64874a709b74d0387e60026ac4387d6/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:52.449543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.449838 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.450113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.478589 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] MFA Rules not processed for user `f64874a709b74d0387e60026ac4387d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:52.479823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:52.565243 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1198/2393] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:52.568695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.642228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2da17e-7b68-4c1e-b9f0-004013411944 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.647182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.647795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.648080 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.648348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.648974 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.651243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.689516 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1196/2394] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:52.694081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.707736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f10ac4-a600-44b5-9f42-39a9343730be None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.774109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.774753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.775290 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.775571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.776202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.778162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.801702 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1199/2395] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:23:52 2022] POST /identity/v3/users => generated 295 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:23:52.805556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.816735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5fd8ac-591f-447f-a1ff-680b896b8692 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.821811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.822522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.822902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.823249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.823953 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.825633 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.840644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1197/2396] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:52.842086 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.852832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197243e-2d79-43e5-8e59-cddda6117108 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.857933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.859456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.859710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.859982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] PATH_INFO: `/v3/projects/0713903344504ebc9e350ea5e2b951fd/users/376caf1be7714a49a723f07e9d725ef5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.880177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0713903344504ebc9e350ea5e2b951fd, user_id=376caf1be7714a49a723f07e9d725ef5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.883317 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:52.897715 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1200/2397] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:52 2022] PUT /identity/v3/projects/0713903344504ebc9e350ea5e2b951fd/users/376caf1be7714a49a723f07e9d725ef5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:52.902041 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:52.920391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834345d2-e583-461a-9bf0-ebcdcf981edc admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:52.994174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:52.994822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:52.995118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:52.995397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:52.996035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:52.998910 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d9fffb-1430-4e9d-8232-27560e4f4546 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:53.015610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1198/2398] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:23:53.020094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:53.034190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e925fa6-bfcf-4e0f-86c2-3213cb434616 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:53.077265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:53.078346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:53.079693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:53.081975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] PATH_INFO: `/v3/projects/0713903344504ebc9e350ea5e2b951fd/users/376caf1be7714a49a723f07e9d725ef5/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:53.085659 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0713903344504ebc9e350ea5e2b951fd, user_id=376caf1be7714a49a723f07e9d725ef5, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:53.089559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:53.109817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1201/2399] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:23:53 2022] PUT /identity/v3/projects/0713903344504ebc9e350ea5e2b951fd/users/376caf1be7714a49a723f07e9d725ef5/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:53.122248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:53.122886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:53.123172 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:53.165354 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] MFA Rules not processed for user `376caf1be7714a49a723f07e9d725ef5`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:53.166594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:53.231120 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:53.262576 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1199/2400] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:53 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:23:53.276563 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:53.278330 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:53.319336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:23:53.351686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd8eddc-8009-4fb3-87e0-d2642929f330 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:53.356152 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:53.356839 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:53.357190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:53.357536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:53.432913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:53.434331 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:53.444373 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1202/2401] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:23:54.985173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:54.995509 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45825eda-241d-4ccf-9fa9-e23a9662e59a None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:54.999075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:55.000347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:55.000628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:55.000902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:55.066711 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:55.068150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:55.077686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1200/2402] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:55.607387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:55.616463 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62b69eb-0b81-4ac7-9158-5bce1cd10548 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:55.620050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:55.620692 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:55.620974 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:55.621248 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:55.630619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:55.632039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b130904-528e-4352-b6ad-069eaa9b567d service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:55.641559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1203/2403] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:23:59.704264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:59.785912 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a0569-7f52-42cf-827b-c77c345b0313 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:23:59.790625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:23:59.791334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:59.791618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:59.791893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] PATH_INFO: `/v3/users/6a6b27aa868a4e32ae7c5f9824854048` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:59.793025 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a6b27aa868a4e32ae7c5f9824854048)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:23:59.794971 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:23:59.863556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a6b27aa868a4e32ae7c5f9824854048'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:59.871408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a6b27aa868a4e32ae7c5f9824854048'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:59.872100 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a6b27aa868a4e32ae7c5f9824854048'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:59.880780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a6b27aa868a4e32ae7c5f9824854048'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:59.881423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a6b27aa868a4e32ae7c5f9824854048'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:23:59.899004 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1201/2404] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:23:59 2022] DELETE /identity/v3/users/6a6b27aa868a4e32ae7c5f9824854048 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:23:59.903985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:23:59.904532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:23:59.904532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:23:59.924714 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:23:59.925903 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:23:59.982227 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1204/2405] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:23:59.992598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:23:59.994018 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:00.004945 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:00.051938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22547a-8ae8-4fe4-bd7f-3fc2100bfcf3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:00.055534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:00.056124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:00.056403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:00.056674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:00.066008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:00.067443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:00.076564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1202/2406] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:23:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:00.350208 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:00.351807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:00.364133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55648644-74ec-44b4-9020-9a78ba05cd05 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:00.368790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:00.369487 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:00.369848 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:00.370188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] PATH_INFO: `/v3/projects/daa30a51af264eddaef749248da416f5` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:00.371402 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=daa30a51af264eddaef749248da416f5)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:00.373353 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:00.421008 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daa30a51af264eddaef749248da416f5', 'request_id': 'req-80a54c10-e003-43a0-87ca-dd074a4e09ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.428300 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daa30a51af264eddaef749248da416f5', 'request_id': 'req-80a54c10-e003-43a0-87ca-dd074a4e09ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.433879 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daa30a51af264eddaef749248da416f5', 'request_id': 'req-80a54c10-e003-43a0-87ca-dd074a4e09ce'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.437417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:00.447641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] The token cache is being invalidate because project daa30a51af264eddaef749248da416f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:00.447939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.449110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.450963 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1205/2407] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:00 2022] DELETE /identity/v3/projects/daa30a51af264eddaef749248da416f5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:00.455160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:00.465202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b03a5b5-5bfb-4880-bbd2-416149bcfd57 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:00.527024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:00.527838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:00.528130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:00.528408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] PATH_INFO: `/v3/users/f64874a709b74d0387e60026ac4387d6` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:00.529770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f64874a709b74d0387e60026ac4387d6)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:00.531735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:00.583883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f64874a709b74d0387e60026ac4387d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.590931 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f64874a709b74d0387e60026ac4387d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.591598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f64874a709b74d0387e60026ac4387d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.600175 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f64874a709b74d0387e60026ac4387d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.601032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f64874a709b74d0387e60026ac4387d6'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:00.625977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1203/2408] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:00 2022] DELETE /identity/v3/users/f64874a709b74d0387e60026ac4387d6 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:24:01.134752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:01.137159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:01.177552 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a54c10-e003-43a0-87ca-dd074a4e09ce admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:01.257570 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:01.258335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:01.261656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:01.261656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] PATH_INFO: `/v3/projects/df8b4755c14f4da5b3828e1d4e50e2d4` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:01.262418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] RBAC: Authorizing `identity:delete_project(project_id=df8b4755c14f4da5b3828e1d4e50e2d4)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:01.266819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:01.329624 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df8b4755c14f4da5b3828e1d4e50e2d4', 'request_id': 'req-ea25c3d7-6815-4a92-ba57-a1c194ae552f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.338346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df8b4755c14f4da5b3828e1d4e50e2d4', 'request_id': 'req-ea25c3d7-6815-4a92-ba57-a1c194ae552f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.344893 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df8b4755c14f4da5b3828e1d4e50e2d4', 'request_id': 'req-ea25c3d7-6815-4a92-ba57-a1c194ae552f'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.360322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] The token cache is being invalidate because project df8b4755c14f4da5b3828e1d4e50e2d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:01.360613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.361396 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.366422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1206/2409] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:01 2022] DELETE /identity/v3/projects/df8b4755c14f4da5b3828e1d4e50e2d4 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:01.368392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:01.369867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:01.385011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1456ed-6dbc-4327-9125-4ba182036d4f admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:01.419751 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:01.468564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:01.470542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:01.470780 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:01.471110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] PATH_INFO: `/v3/users/376caf1be7714a49a723f07e9d725ef5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:01.472985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] RBAC: Authorizing `identity:delete_user(user_id=376caf1be7714a49a723f07e9d725ef5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:01.475604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:01.529174 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:01.530736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:01.545655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '376caf1be7714a49a723f07e9d725ef5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.556365 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '376caf1be7714a49a723f07e9d725ef5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.557250 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '376caf1be7714a49a723f07e9d725ef5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.567620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '376caf1be7714a49a723f07e9d725ef5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.568667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '376caf1be7714a49a723f07e9d725ef5'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:01.589913 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1204/2410] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:01 2022] DELETE /identity/v3/users/376caf1be7714a49a723f07e9d725ef5 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:01.631957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea25c3d7-6815-4a92-ba57-a1c194ae552f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:01.655048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:01.714133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '8060598d22754154a1c7af65a96aaac2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:01.714869 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:01.715156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:01.715453 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:01.717301 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:01.730865 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:01.732528 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27b7416-c381-41cb-80e3-810070547d26 service nova] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:01.744473 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1207/2411] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:24:01.899608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:01.901142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:01.985778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f5e476-d90f-4d90-96d5-9fa5f38f9979 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:01.990558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:01.991192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:01.991479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:01.991758 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] PATH_INFO: `/v3/projects/0713903344504ebc9e350ea5e2b951fd` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:01.992939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0713903344504ebc9e350ea5e2b951fd)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:01.995014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.048392 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0713903344504ebc9e350ea5e2b951fd', 'request_id': 'req-64fe350e-6abf-4951-a2db-95c5c6432e56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:02.049526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:02.056258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0713903344504ebc9e350ea5e2b951fd', 'request_id': 'req-64fe350e-6abf-4951-a2db-95c5c6432e56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:02.063867 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0713903344504ebc9e350ea5e2b951fd', 'request_id': 'req-64fe350e-6abf-4951-a2db-95c5c6432e56'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:02.078089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] The token cache is being invalidate because project 0713903344504ebc9e350ea5e2b951fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:02.078735 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:02.079339 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-64fe350e-6abf-4951-a2db-95c5c6432e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:02.081859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1205/2412] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:01 2022] DELETE /identity/v3/projects/0713903344504ebc9e350ea5e2b951fd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:02.086488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.086721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.086993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.108235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:02.109457 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:02.195831 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1208/2413] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:02.203922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.204210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.204485 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.228428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:02.229644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:02.238755 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1206/2414] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:02.243515 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.254571 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28007fa-d773-432e-890e-ff2ad2468012 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.260403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.261303 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.261763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.262306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.263215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:24:02.263512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.265190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.287144 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1209/2415] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:24:02.288358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.297829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e6dc35-e740-4b61-8744-26378fc7df2d None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.303253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.303864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.304140 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.305829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.305829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.307465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.354969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1207/2416] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:02.360039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.376403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920c8021-4c79-4e8a-ae80-24fe65370051 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.442327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.442976 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.443765 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.444014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.444693 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.446638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.469967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1210/2417] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:02.473983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.484171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26ec4c9-dee8-492e-bd93-1d35e22d8ae5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.489454 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.490051 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.490332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.490658 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.491289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.492880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.503883 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1208/2418] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:02.508081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.519564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8e490b-a54d-4e39-aa6d-20f4a567ba36 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.531074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.531754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.532098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.533825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.533825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.536007 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f86d46c-8e81-48b2-ac86-e208cc667c91 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.547339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1211/2419] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:02.551676 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.561494 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ab2827-d1a9-4610-a465-1087dbc3e30e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.566674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.567342 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.567618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.567888 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] PATH_INFO: `/v3/projects/9258b768f0194c078659f614a8bec6e7/users/9816a809c2b84f37a766d30d870c1c15/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.582877 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9258b768f0194c078659f614a8bec6e7, user_id=9816a809c2b84f37a766d30d870c1c15, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.585518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.599156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1209/2420] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:02 2022] PUT /identity/v3/projects/9258b768f0194c078659f614a8bec6e7/users/9816a809c2b84f37a766d30d870c1c15/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:02.665262 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.665547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.665817 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.695233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] MFA Rules not processed for user `9816a809c2b84f37a766d30d870c1c15`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:02.696426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:02.700276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:02.761859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1212/2421] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:02.766308 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.827344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a551857e-c72a-47e1-a869-1bc816be9077 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.832117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.832722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.832999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.833269 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.833903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.835809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.862272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1210/2422] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:02.867067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.877491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88fe7ae-db0b-442b-8e37-b2b8e0a16a57 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.932609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.933206 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.933483 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.933762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.934397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.936313 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.956861 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1213/2423] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:02 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:02.960929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:02.971176 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f961e-974e-47ca-98fe-8f32ad7fb3cb admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:02.976356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:02.976946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:02.977228 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:02.977504 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:02.978163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:02.979801 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:02.990060 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1211/2424] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:02.993945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.003815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b76397-a708-40d9-be82-60b939b27b0c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.008399 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.008983 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.009263 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.009537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.010165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.011810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a5b548-aff7-4555-9f09-56f7f8d5d7b1 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.022036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1214/2425] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:03.025979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.036657 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364c62d-418b-4817-8600-905cc146fa5a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.041194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.041828 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.042132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.042390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] PATH_INFO: `/v3/projects/94480d4b82d645668c93896e4f045f45/users/8d08b864892b414fa4cad58675fe604f/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.057061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94480d4b82d645668c93896e4f045f45, user_id=8d08b864892b414fa4cad58675fe604f, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.059777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.072810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1212/2426] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:03 2022] PUT /identity/v3/projects/94480d4b82d645668c93896e4f045f45/users/8d08b864892b414fa4cad58675fe604f/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:03.083449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.083739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.084014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.111787 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] MFA Rules not processed for user `8d08b864892b414fa4cad58675fe604f`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:03.113084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:03.176832 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1215/2427] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:03.181337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.209124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:03.245289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c382c0a4-7134-4030-95de-6107fbd95e18 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.249906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.250542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.250827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.251101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.251743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.253655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.279905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1213/2428] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:03 2022] POST /identity/v3/projects => generated 385 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:03.283939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.295140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f7f7ff-34b4-435c-a09c-cfda6a31c203 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.303926 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:03.351061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.351691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.351971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.352245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.352922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.354946 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.376249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1216/2429] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:03 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:03.380841 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.391162 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ac154-4cc6-4d7f-9981-f2904c8cb0a5 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.404677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.405287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.405569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.405844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.406511 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.408417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.419396 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1214/2430] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:03.423653 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.433660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0ca76e-e6bf-43fb-a3ba-1b2a147516ee admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.438280 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.438956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.439238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.439512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] PATH_INFO: `/v3/projects/b2035b5a432a4beabb0d0374495a7549/users/48132928f6324651bc20e588f7176bea/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.454378 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2035b5a432a4beabb0d0374495a7549, user_id=48132928f6324651bc20e588f7176bea, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.457690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.470355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1217/2431] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:03 2022] PUT /identity/v3/projects/b2035b5a432a4beabb0d0374495a7549/users/48132928f6324651bc20e588f7176bea/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:03.474320 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.484305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0eca49-c03c-42ff-955c-589b9a61f7e6 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.541921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.542563 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.542845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.543118 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.543803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.545445 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497e3b21-6633-4307-a189-e565e0b2403e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.556832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1215/2432] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:03.561162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.572168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e13122-4002-40ee-ae96-946ffe22f964 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.577090 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.577756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.578127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.578522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] PATH_INFO: `/v3/projects/b2035b5a432a4beabb0d0374495a7549/users/48132928f6324651bc20e588f7176bea/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.580551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2035b5a432a4beabb0d0374495a7549, user_id=48132928f6324651bc20e588f7176bea, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.593411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.605215 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1218/2433] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:03 2022] PUT /identity/v3/projects/b2035b5a432a4beabb0d0374495a7549/users/48132928f6324651bc20e588f7176bea/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:03.616093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.616559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.616970 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.646680 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] MFA Rules not processed for user `48132928f6324651bc20e588f7176bea`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:03.648180 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:03.726826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1216/2434] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:03 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:03.738345 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:03.740260 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:03.815601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2824c0-2222-454f-95e9-6f7714cc9761 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:03.819255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:03.819945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:03.820121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:03.820444 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:03.879327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:03.880747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:03.889420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1219/2435] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:05.404648 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:05.406057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:05.418455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc42684-4c72-404c-ad4e-c91b19a95ebb None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:05.422111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:05.422775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:05.423182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:05.423488 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:05.441264 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:05.482930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:05.484358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea7d483-35c4-4fe7-b3e6-936b2a680b57 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:05.493859 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1217/2436] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:05 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:06.055224 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:06.080583 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c4885e-0970-4cd8-a960-2891b29ea24e service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:06.092709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:06.093793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:06.093793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:06.094018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:06.113927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:06.115952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:06.127980 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1220/2437] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:06 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:08.647443 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:08.648218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:08.648218 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:08.680592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] MFA Rules not processed for user `768efb731b82485ba4e0a51901710a16`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:08.682592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:08.784399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1218/2438] 10.222.0.23 () {58 vars in 967 bytes} [Sun Oct 23 02:24:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:08.797985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:08.799525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:08.811173 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:08.885558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d55f74-40f6-46fc-8401-767e8f5617e7 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:08.890558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e03f5a82065a41508004af126326e8ca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:08.891231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:08.891576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:08.891918 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:08.903099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:08.905261 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa743f5-fde1-4106-8230-f39c03a02ac3 service swift] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:08.916226 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1221/2439] 10.222.0.23 () {60 vars in 1384 bytes} [Sun Oct 23 02:24:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Oct 23 02:24:10.265103 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:10.342822 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd3bf23-1c18-4569-abcd-5c2bb658ca19 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:10.350052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:10.351242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:10.351750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:10.352406 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] PATH_INFO: `/v3/users/9816a809c2b84f37a766d30d870c1c15` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:10.353821 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9816a809c2b84f37a766d30d870c1c15)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:10.355977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:10.435432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9816a809c2b84f37a766d30d870c1c15'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:10.445496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9816a809c2b84f37a766d30d870c1c15'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:10.446397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9816a809c2b84f37a766d30d870c1c15'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:10.463196 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9816a809c2b84f37a766d30d870c1c15'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:10.463988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9816a809c2b84f37a766d30d870c1c15'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:10.467299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:10.493015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1219/2440] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:10 2022] DELETE /identity/v3/users/9816a809c2b84f37a766d30d870c1c15 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:10.498013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:10.498636 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:10.499236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:10.527664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:10.529278 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:10.613663 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1222/2441] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:10.624980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:10.626568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:10.714034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaea5b9-780d-4bb0-bc80-dce9230f3104 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:10.718860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:10.719489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:10.719919 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:10.720202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:10.730788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:10.732244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:10.744423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1220/2442] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:11.240321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:11.242535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:11.257931 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43134b14-4dab-4b03-902c-26f6feed0f8c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:11.263480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:11.264391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:11.264840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:11.265275 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] PATH_INFO: `/v3/projects/9258b768f0194c078659f614a8bec6e7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:11.267387 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9258b768f0194c078659f614a8bec6e7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:11.277897 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:11.356682 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9258b768f0194c078659f614a8bec6e7', 'request_id': 'req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.363989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9258b768f0194c078659f614a8bec6e7', 'request_id': 'req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.371513 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9258b768f0194c078659f614a8bec6e7', 'request_id': 'req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.387656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] The token cache is being invalidate because project 9258b768f0194c078659f614a8bec6e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:11.388482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.389142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.390947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1223/2443] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:11 2022] DELETE /identity/v3/projects/9258b768f0194c078659f614a8bec6e7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:11.394975 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:11.409655 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e74932f-5304-434c-ae46-e54850c594f7 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:11.509835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:11.510526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:11.511783 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:11.512109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] PATH_INFO: `/v3/users/8d08b864892b414fa4cad58675fe604f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:11.513448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d08b864892b414fa4cad58675fe604f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:11.516594 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:11.610984 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d08b864892b414fa4cad58675fe604f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.621203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d08b864892b414fa4cad58675fe604f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.622296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d08b864892b414fa4cad58675fe604f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.634335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d08b864892b414fa4cad58675fe604f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.635285 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d08b864892b414fa4cad58675fe604f'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:11.667413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1221/2444] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:11 2022] DELETE /identity/v3/users/8d08b864892b414fa4cad58675fe604f => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:11.983777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:11.987445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:12.045558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:12.108982 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6032ef3-9c8c-4256-8ef6-5e158d1baf99 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:12.183905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:12.186188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:12.202992 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:12.204236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:12.204853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:12.205232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:12.206551 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:12.211251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:12.310508 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:12.312334 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:12.325050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1224/2445] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:12.339625 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e93cf1-33a7-44a1-82ff-322ca3397eec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:12.347946 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:12.349383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:12.349815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:12.351675 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] PATH_INFO: `/v3/projects/94480d4b82d645668c93896e4f045f45` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:12.353838 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94480d4b82d645668c93896e4f045f45)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:12.357040 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:12.423519 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94480d4b82d645668c93896e4f045f45', 'request_id': 'req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.432432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94480d4b82d645668c93896e4f045f45', 'request_id': 'req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.442119 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94480d4b82d645668c93896e4f045f45', 'request_id': 'req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.464330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] The token cache is being invalidate because project 94480d4b82d645668c93896e4f045f45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:12.464640 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.465315 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.467266 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1222/2446] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:12 2022] DELETE /identity/v3/projects/94480d4b82d645668c93896e4f045f45 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:12.471628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:12.483480 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251d01dc-6a5d-471e-a400-9ae635ba071e service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:12.563763 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:12.564171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:12.564698 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:12.565114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] PATH_INFO: `/v3/users/48132928f6324651bc20e588f7176bea` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:12.567622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] RBAC: Authorizing `identity:delete_user(user_id=48132928f6324651bc20e588f7176bea)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:12.570737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:12.638567 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:12.684406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48132928f6324651bc20e588f7176bea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.700913 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48132928f6324651bc20e588f7176bea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.714917 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48132928f6324651bc20e588f7176bea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.715939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48132928f6324651bc20e588f7176bea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.716810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-470d68e1-81eb-4211-8df0-45775504e54e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48132928f6324651bc20e588f7176bea'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:12.746168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1225/2447] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:12 2022] DELETE /identity/v3/users/48132928f6324651bc20e588f7176bea => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:13.250756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.252712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:13.277763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cd3-0e48-4ff3-9f3f-0b499b728a71 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:13.360856 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:13.361637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.361985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.362328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] PATH_INFO: `/v3/projects/b2035b5a432a4beabb0d0374495a7549` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.363577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2035b5a432a4beabb0d0374495a7549)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:13.365584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:13.405855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:13.417094 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2035b5a432a4beabb0d0374495a7549', 'request_id': 'req-2201a253-ef69-43f4-ab24-0ecf8871f403'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:13.430211 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2035b5a432a4beabb0d0374495a7549', 'request_id': 'req-2201a253-ef69-43f4-ab24-0ecf8871f403'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:13.436257 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2035b5a432a4beabb0d0374495a7549', 'request_id': 'req-2201a253-ef69-43f4-ab24-0ecf8871f403'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:13.454350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] The token cache is being invalidate because project b2035b5a432a4beabb0d0374495a7549 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:13.454688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:13.455401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2201a253-ef69-43f4-ab24-0ecf8871f403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:13.457226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1223/2448] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:13 2022] DELETE /identity/v3/projects/b2035b5a432a4beabb0d0374495a7549 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:13.462033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.462588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.462745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.483259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:13.484733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:13.554052 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1226/2449] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:13.561872 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6516b-6141-4738-9775-d75622527f28 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.562399 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6516b-6141-4738-9775-d75622527f28 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.562728 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6516b-6141-4738-9775-d75622527f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.586470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-efd6516b-6141-4738-9775-d75622527f28 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:13.588009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-efd6516b-6141-4738-9775-d75622527f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:13.602107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1224/2450] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:24:13.603625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.616607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fab19-0d2b-4e3d-b10a-65f3b7e92d0f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:13.622124 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:13.622993 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.623853 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.624196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.624910 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:24:13.625320 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:13.628214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:13.650620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1227/2451] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:24:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:24:13.653075 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6516b-6141-4738-9775-d75622527f28 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.662968 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6516b-6141-4738-9775-d75622527f28 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:13.668117 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:13.668730 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.669016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.669295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.670514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:13.674545 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:13.744200 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1225/2452] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:13 2022] POST /identity/v3/projects => generated 383 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:24:13.746236 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.761417 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da6765-50e2-4dfe-8be8-fa7762d48c5e admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:13.872421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:13.873114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.873422 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.873694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.874595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:13.876706 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:13.906525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1228/2453] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:13 2022] POST /identity/v3/users => generated 292 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:13.908109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.923238 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd8284d-8b53-45dc-b985-c151522b716c admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:13.959477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:13.960477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:13.960916 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:13.962222 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:13.963131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:13.964942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:13.983667 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1226/2454] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:24:13.987759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:13.999562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6f38d6-1168-4513-91d9-a5fcd5ab6212 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:14.005110 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:14.006478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.006478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.006478 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.007017 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:14.008967 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf90e0e-fd5d-44d8-a834-c099b2e68630 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:14.023462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1229/2455] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:14.024729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:14.035944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b01cc7-9f90-441c-bbab-28d80a25f08b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:14.044182 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:14.044860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.045142 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.045416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] PATH_INFO: `/v3/projects/3f32b2691a2244adbaf9a37f6981b393/users/4f7ae78b38be4dee9ee5b695b05c2e3a/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.061813 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f32b2691a2244adbaf9a37f6981b393, user_id=4f7ae78b38be4dee9ee5b695b05c2e3a, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:14.064584 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:14.081083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1227/2456] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:14 2022] PUT /identity/v3/projects/3f32b2691a2244adbaf9a37f6981b393/users/4f7ae78b38be4dee9ee5b695b05c2e3a/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:14.087080 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.087374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.087651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.108498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:14.109717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.115534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:14.183496 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1230/2457] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:24:14.195957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:14.197989 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.283803 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74376c8f-1179-49e6-b9b6-f0ac90133288 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:14.289639 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:14.290350 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.290842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.291131 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.315032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:14.316794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:14.332793 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1228/2458] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:14.475358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:14.477757 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.498098 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b03d81-829a-44d3-8c27-82db7bef4bc1 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:14.510342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:14.511734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.512082 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.512488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.517077 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.534822 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:14.538634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23892eee-f8c6-4319-b5a2-7fe42e2ff4f0 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:14.550016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:14.551983 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.561872 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1231/2459] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:14.710387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc05d149-8e31-4a53-aee9-a13fffe64ce3 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:14.718717 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:14.719538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:14.720071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:14.720355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:14.723432 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:14.735011 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:14.746686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:14.749806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:14.786468 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1229/2460] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:24:16.005055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:16.005603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:16.005603 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:16.056235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] MFA Rules not processed for user `4f7ae78b38be4dee9ee5b695b05c2e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:16.060594 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:16.175368 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1232/2461] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:16 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:16.185312 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:16.187311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:16.277486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523f9f66-d907-445d-895f-8ccc529370fa service placement] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:16.281668 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:16.282836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:16.283589 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:16.284078 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:16.297171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:16.299309 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42122a5b-2591-4a76-9132-95e9c651a68b service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:16.314397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1230/2462] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:16.401681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:16.416136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6c27c-8009-42dc-8719-3b00057fc1af None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:16.422286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:16.423527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:16.423964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:16.424990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:16.427641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:16.446642 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:16.450247 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:16.464183 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1233/2463] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:17.926537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:17.927839 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:17.928453 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:17.965707 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] MFA Rules not processed for user `4f7ae78b38be4dee9ee5b695b05c2e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:17.967083 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:17.986272 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1231/2464] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:24:18.456829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:18.484325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed1ed67-7e53-4ac4-89d5-b772efa2242d service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:18.492915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:18.494048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:18.495327 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:18.495958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:18.499218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:18.521347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:18.523435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:18.540100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1234/2465] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:19.200134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:19.210337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:19.259061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39539d1-76cf-4f9b-be26-53b4bfa74df1 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:19.270792 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:19.272204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:19.272663 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:19.273938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] PATH_INFO: `/v3/users/66975cf6d8fa417e8df429d4c89ef910` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:19.275723 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] RBAC: Authorizing `identity:delete_user(user_id=66975cf6d8fa417e8df429d4c89ef910)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:19.278130 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:19.351555 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66975cf6d8fa417e8df429d4c89ef910'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.364301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66975cf6d8fa417e8df429d4c89ef910'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.365347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66975cf6d8fa417e8df429d4c89ef910'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.380244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66975cf6d8fa417e8df429d4c89ef910'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.381167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-11abbd81-67ca-448c-918e-74341170668c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66975cf6d8fa417e8df429d4c89ef910'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.429886 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1232/2466] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:19 2022] DELETE /identity/v3/users/66975cf6d8fa417e8df429d4c89ef910 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:19.671070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:19.701609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acee53d-da84-4c8c-a6b4-b6dffd8cfd99 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:19.875114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:19.876846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:19.877310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:19.877769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] PATH_INFO: `/v3/projects/eab81e47f8d6426fbd5b50d1eb7fc14a` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:19.880018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab81e47f8d6426fbd5b50d1eb7fc14a)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:19.882268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:19.886171 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=100 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:19.970233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab81e47f8d6426fbd5b50d1eb7fc14a', 'request_id': 'req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.978747 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab81e47f8d6426fbd5b50d1eb7fc14a', 'request_id': 'req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:19.985827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab81e47f8d6426fbd5b50d1eb7fc14a', 'request_id': 'req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:20.017013 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] The token cache is being invalidate because project eab81e47f8d6426fbd5b50d1eb7fc14a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:20.017516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:20.018491 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-094c3457-ec6e-4888-bd6e-aa7e2a277a3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:20.026140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1235/2467] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:19 2022] DELETE /identity/v3/projects/eab81e47f8d6426fbd5b50d1eb7fc14a => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:20.028794 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.029088 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.029367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.073561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:20.075052 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:20.201817 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1233/2468] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:20.209979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.210298 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.210750 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.252283 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:20.254064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:20.291279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1236/2469] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:20.292744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.299105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:20.308194 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e282906e-d4c4-4765-846a-946f3d16e077 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.319305 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.319938 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.320357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.320682 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.321475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:24:20.321786 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.323492 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.364174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1234/2470] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:24:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:24:20.365962 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.389354 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e7ea96-38e3-4490-9f20-93899b389ee3 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.409310 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.410339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.410801 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.411232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.412259 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.414352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.478503 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1237/2471] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:20 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:20.482922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.494224 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991d8352-5cd7-43d5-87e1-caf52773bee3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.596553 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.597745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.599032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.599471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.600440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.603255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.644727 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1235/2472] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:20 2022] POST /identity/v3/users => generated 287 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:20.646958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.662597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d96fa3b-e7c6-45fb-8422-3b5d19b041f8 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.668877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.669655 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.670034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.670827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.671189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.672909 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.687874 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1238/2473] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:20.689390 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.718328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37168b86-1152-4296-a186-0a0a4eecd3ec admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.739467 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.740441 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.740851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.741829 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.742795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.745278 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe5f45e-e367-4ffb-8755-ef67290c4484 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.779510 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1236/2474] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:20.781003 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:20.799792 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd4d359-c99e-4e16-be5c-a831a62a2007 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:20.812393 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:20.813430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.813911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.814664 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] PATH_INFO: `/v3/projects/87abc858941b48deb15d412a6c05c155/users/52d70176fa0d4021b85054b957e7b355/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.836579 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=87abc858941b48deb15d412a6c05c155, user_id=52d70176fa0d4021b85054b957e7b355, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:20.841437 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:20.867127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1239/2475] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:20 2022] PUT /identity/v3/projects/87abc858941b48deb15d412a6c05c155/users/52d70176fa0d4021b85054b957e7b355/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:20.879716 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:20.880295 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:20.880712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:20.949059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] MFA Rules not processed for user `52d70176fa0d4021b85054b957e7b355`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:20.950644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:21.121212 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.122255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1237/2476] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:24:21.228937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078afdf9-187a-407b-889b-7df137819fb4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.235367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.236225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.236600 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.237036 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.238521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.241968 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.332325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1240/2477] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:24:21 2022] POST /identity/v3/projects => generated 371 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:21.337980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.352020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce5590c-593c-4c41-9cb4-beda079aca82 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.407225 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:21.414393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.419738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.420034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.420397 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.421093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.425558 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.456360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1238/2478] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:24:21 2022] POST /identity/v3/users => generated 292 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:24:21.463574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.463761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.467244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:21.475768 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa266-a8ea-42a4-b33c-d566a3be06b2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.482050 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.490752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.491265 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.491744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.492643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.495205 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.498208 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5872a1eb-bec4-4008-9d04-f74358fcf2ac admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.503093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.503734 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.504016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.504292 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] PATH_INFO: `/v3/users/7a3de8194ccd4eec9c19a6b250944457` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.512641 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1241/2479] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:21.520321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.523015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a3de8194ccd4eec9c19a6b250944457)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.532000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.547601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:21.563071 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3559183d-2e71-4ee9-92ed-50085910e86f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.572020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.573440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.573733 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.574536 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] PATH_INFO: `/v3/projects/5afbe7e3fc474c1b9519be102c42d011/users/b7ad52a082d0447ab8313e3ee1a65469/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.613620 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afbe7e3fc474c1b9519be102c42d011, user_id=b7ad52a082d0447ab8313e3ee1a65469, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.623391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.656677 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1242/2480] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:21 2022] PUT /identity/v3/projects/5afbe7e3fc474c1b9519be102c42d011/users/b7ad52a082d0447ab8313e3ee1a65469/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:21.659000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.692815 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a3de8194ccd4eec9c19a6b250944457'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:21.709503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a3de8194ccd4eec9c19a6b250944457'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:21.710562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a3de8194ccd4eec9c19a6b250944457'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:21.754939 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a3de8194ccd4eec9c19a6b250944457'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:21.755991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a3de8194ccd4eec9c19a6b250944457'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:21.798600 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1239/2481] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:21 2022] DELETE /identity/v3/users/7a3de8194ccd4eec9c19a6b250944457 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:24:21.804638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a530db5-dddf-4527-a8d4-d29aa9b2c51a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.870654 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.871401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.871687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.871958 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.872601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.874510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9322b3-2c0a-4ea6-918c-0604b4f6cfd0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.889233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1243/2482] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:24:21 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:24:21.894518 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:21.896114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:21.910181 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c075ebb-c45e-4842-bb12-0f4497d0e484 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:21.917766 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:21.918495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.918827 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.919109 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] PATH_INFO: `/v3/projects/5afbe7e3fc474c1b9519be102c42d011/users/b7ad52a082d0447ab8313e3ee1a65469/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.921364 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5afbe7e3fc474c1b9519be102c42d011, user_id=b7ad52a082d0447ab8313e3ee1a65469, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:21.923896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:21.945566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1240/2483] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:24:21 2022] PUT /identity/v3/projects/5afbe7e3fc474c1b9519be102c42d011/users/b7ad52a082d0447ab8313e3ee1a65469/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:21.957924 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:21.958246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:21.958517 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:21.990560 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] MFA Rules not processed for user `b7ad52a082d0447ab8313e3ee1a65469`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:24:21.991471 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:22.148105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1244/2484] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:24:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:24:22.161385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:22.168947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:22.281255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b1cbe-510f-4621-ae90-86e694275ba0 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:22.285383 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:22.286002 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:22.286286 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:22.286595 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:22.310654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:22.313297 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:22.342922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1241/2485] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:22 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:22.466207 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:22.480630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23b61d-c2c9-486a-ab97-4a70fba0a48c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:22.490781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:22.491406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:22.491688 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:22.491964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:22.499408 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:22.509736 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:22.546905 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:22.548385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:22.623572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:22.625182 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:22.649101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1245/2486] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:24:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:24:22.661577 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:22.686129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6e128-f40f-487e-bd46-43cff37cb026 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:22.701173 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:22.701954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:22.702335 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:22.702781 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] PATH_INFO: `/v3/projects/7d85fefc9f0b47f8bad45fb6dd0810f9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:22.704107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d85fefc9f0b47f8bad45fb6dd0810f9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:22.706171 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:22.782263 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d85fefc9f0b47f8bad45fb6dd0810f9', 'request_id': 'req-5b1e486a-a03f-43a1-acad-37270f4999e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:22.809677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d85fefc9f0b47f8bad45fb6dd0810f9', 'request_id': 'req-5b1e486a-a03f-43a1-acad-37270f4999e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:22.845804 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d85fefc9f0b47f8bad45fb6dd0810f9', 'request_id': 'req-5b1e486a-a03f-43a1-acad-37270f4999e7'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:22.889236 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] The token cache is being invalidate because project 7d85fefc9f0b47f8bad45fb6dd0810f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:22.889536 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:22.890465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:22.892596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1242/2487] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:22 2022] DELETE /identity/v3/projects/7d85fefc9f0b47f8bad45fb6dd0810f9 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:22.896573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:22.918456 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f8d0db-c2dd-4dcf-b6de-2e082bbc3613 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:22.989811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:22.990626 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:22.991041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:22.991601 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] PATH_INFO: `/v3/users/e43c3e4d87e844b3aa9b3ce31155b802` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:22.993100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] RBAC: Authorizing `identity:delete_user(user_id=e43c3e4d87e844b3aa9b3ce31155b802)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:22.995433 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:23.050064 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e43c3e4d87e844b3aa9b3ce31155b802'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.058851 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e43c3e4d87e844b3aa9b3ce31155b802'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.068580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e43c3e4d87e844b3aa9b3ce31155b802'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.069284 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e43c3e4d87e844b3aa9b3ce31155b802'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.070752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e43c3e4d87e844b3aa9b3ce31155b802'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.107322 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1246/2488] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:22 2022] DELETE /identity/v3/users/e43c3e4d87e844b3aa9b3ce31155b802 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:24:23.462132 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:23.463623 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:23.478415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e486a-a03f-43a1-acad-37270f4999e7 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:23.565947 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:23.568659 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:23.569057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:23.569754 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] PATH_INFO: `/v3/projects/29a8b1e914054f14a418ed039f8a1a4f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:23.575059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a8b1e914054f14a418ed039f8a1a4f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:23.578160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:23.634944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a8b1e914054f14a418ed039f8a1a4f', 'request_id': 'req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.643988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a8b1e914054f14a418ed039f8a1a4f', 'request_id': 'req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.651301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a8b1e914054f14a418ed039f8a1a4f', 'request_id': 'req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.670256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] The token cache is being invalidate because project 29a8b1e914054f14a418ed039f8a1a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:23.671047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.672112 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:23.674229 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1243/2489] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:23 2022] DELETE /identity/v3/projects/29a8b1e914054f14a418ed039f8a1a4f => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:26.428421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:26.429770 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:26.445521 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d54ece-764c-40f2-8e01-c32f4b9a75ae admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:26.467282 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:26.505492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:26.506119 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:26.506394 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:26.506695 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] PATH_INFO: `/v3/users/d8cd3210dbea46d6912c93c6d28356ae` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:26.521034 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8cd3210dbea46d6912c93c6d28356ae)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:26.522979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:26.568403 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8cd3210dbea46d6912c93c6d28356ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.575796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8cd3210dbea46d6912c93c6d28356ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.586397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8cd3210dbea46d6912c93c6d28356ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.587185 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8cd3210dbea46d6912c93c6d28356ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.587742 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8cd3210dbea46d6912c93c6d28356ae'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.612155 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1247/2490] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:26 2022] DELETE /identity/v3/users/d8cd3210dbea46d6912c93c6d28356ae => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:26.790775 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:26.792837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:26.806627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ce8960-74f5-4b7b-9787-762b7b3ca19d admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:26.812712 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:26.891063 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:26.891694 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:26.891977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:26.892252 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] PATH_INFO: `/v3/projects/10f27676e78f475fa9734f3d6f541da5` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:26.902964 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10f27676e78f475fa9734f3d6f541da5)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:26.904866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:26.950957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10f27676e78f475fa9734f3d6f541da5', 'request_id': 'req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.957614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10f27676e78f475fa9734f3d6f541da5', 'request_id': 'req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.963378 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10f27676e78f475fa9734f3d6f541da5', 'request_id': 'req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.975554 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] The token cache is being invalidate because project 10f27676e78f475fa9734f3d6f541da5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:26.975835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.976463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:26.977902 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1244/2491] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:26 2022] DELETE /identity/v3/projects/10f27676e78f475fa9734f3d6f541da5 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:26.982015 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:26.983547 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:26.996943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08a664-24b5-4942-8c9e-f0a9fe2a459c admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:27.051367 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:27.051987 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:27.052267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:27.052542 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] PATH_INFO: `/v3/users/3e4d471ec3dc43e08197dab0c9ec8ca7` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:27.066268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e4d471ec3dc43e08197dab0c9ec8ca7)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:27.068276 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:27.090113 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:27.115535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e4d471ec3dc43e08197dab0c9ec8ca7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.122952 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e4d471ec3dc43e08197dab0c9ec8ca7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.130195 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e4d471ec3dc43e08197dab0c9ec8ca7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.130864 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e4d471ec3dc43e08197dab0c9ec8ca7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.131477 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e4d471ec3dc43e08197dab0c9ec8ca7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.149252 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1248/2492] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:24:26 2022] DELETE /identity/v3/users/3e4d471ec3dc43e08197dab0c9ec8ca7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:27.319831 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:24:27.321441 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:24:27.335012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a09bd3-aa30-4d4a-b4c9-f0790fb07552 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:24:27.393525 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:24:27.394157 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:24:27.394463 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:24:27.394750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] PATH_INFO: `/v3/projects/94ac7006ccce45c19464a0adfbe41af9` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:24:27.405490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ac7006ccce45c19464a0adfbe41af9)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:24:27.407435 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:24:27.431992 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:27.457368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ac7006ccce45c19464a0adfbe41af9', 'request_id': 'req-bcf5d7c7-a039-4321-981d-7e361c07e7b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.464125 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ac7006ccce45c19464a0adfbe41af9', 'request_id': 'req-bcf5d7c7-a039-4321-981d-7e361c07e7b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.472389 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ac7006ccce45c19464a0adfbe41af9', 'request_id': 'req-bcf5d7c7-a039-4321-981d-7e361c07e7b1'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.498534 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] The token cache is being invalidate because project 94ac7006ccce45c19464a0adfbe41af9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:24:27.498811 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.499318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bcf5d7c7-a039-4321-981d-7e361c07e7b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:24:27.500797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1245/2493] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:24:27 2022] DELETE /identity/v3/projects/94ac7006ccce45c19464a0adfbe41af9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:24:37.145814 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=20,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:24:37.494636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:10.791500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:10.793114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:10.806100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7ad1b-2e6f-4d26-a5db-882dff5ff2f0 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:10.865769 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:10.866429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:10.866737 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:10.867118 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] PATH_INFO: `/v3/users/52d70176fa0d4021b85054b957e7b355` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:10.868268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=52d70176fa0d4021b85054b957e7b355)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:10.870729 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:10.917493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52d70176fa0d4021b85054b957e7b355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:10.924519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52d70176fa0d4021b85054b957e7b355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:10.931722 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52d70176fa0d4021b85054b957e7b355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:10.932969 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52d70176fa0d4021b85054b957e7b355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:10.933574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52d70176fa0d4021b85054b957e7b355'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:10.950947 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1249/2494] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:10 2022] DELETE /identity/v3/users/52d70176fa0d4021b85054b957e7b355 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:10.955438 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:10.956193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:10.956375 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:10.975124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:10.976299 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:11.034763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1246/2495] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:11.045502 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:11.046911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:11.102556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86080f4-17ab-4809-99df-0073c47408cc admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:11.107021 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:11.107608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:11.107889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:11.108159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:11.116846 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:11.118257 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:11.128130 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1250/2496] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:11.403535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:11.405059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:11.417967 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90e7334-900d-4659-aa60-0a2cb5590950 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:11.422900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:11.423566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:11.423979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:11.424163 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] PATH_INFO: `/v3/projects/87abc858941b48deb15d412a6c05c155` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:11.425287 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87abc858941b48deb15d412a6c05c155)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:11.427210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:11.473977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87abc858941b48deb15d412a6c05c155', 'request_id': 'req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.480842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87abc858941b48deb15d412a6c05c155', 'request_id': 'req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.486966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87abc858941b48deb15d412a6c05c155', 'request_id': 'req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.500493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] The token cache is being invalidate because project 87abc858941b48deb15d412a6c05c155 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:11.500765 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.501386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.502893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1247/2497] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:11 2022] DELETE /identity/v3/projects/87abc858941b48deb15d412a6c05c155 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:11.506856 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:11.517546 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1ae9dd-946b-4f34-a969-e5f89335386a service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:11.535745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:11.576339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:11.576955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:11.577233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:11.577507 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] PATH_INFO: `/v3/users/b7ad52a082d0447ab8313e3ee1a65469` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:11.578638 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7ad52a082d0447ab8313e3ee1a65469)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:11.580534 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:11.626978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7ad52a082d0447ab8313e3ee1a65469'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.633725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7ad52a082d0447ab8313e3ee1a65469'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.641523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7ad52a082d0447ab8313e3ee1a65469'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.642136 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7ad52a082d0447ab8313e3ee1a65469'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.642760 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-816da4a4-f3eb-42f5-8bf0-8f3c45894873 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7ad52a082d0447ab8313e3ee1a65469'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:11.660516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1251/2498] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:11 2022] DELETE /identity/v3/users/b7ad52a082d0447ab8313e3ee1a65469 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:11.929843 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:11.931298 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:11.944047 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b360a-4f4d-4bbe-ae4d-3c3e64a7b1d8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.000099 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.000802 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.001003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.001279 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] PATH_INFO: `/v3/projects/5afbe7e3fc474c1b9519be102c42d011` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.002401 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5afbe7e3fc474c1b9519be102c42d011)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.004329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.034777 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:12.052497 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5afbe7e3fc474c1b9519be102c42d011', 'request_id': 'req-cbab9a2f-cc38-43cf-beb8-07184024d9d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:12.058743 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5afbe7e3fc474c1b9519be102c42d011', 'request_id': 'req-cbab9a2f-cc38-43cf-beb8-07184024d9d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:12.064210 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5afbe7e3fc474c1b9519be102c42d011', 'request_id': 'req-cbab9a2f-cc38-43cf-beb8-07184024d9d4'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:12.077001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] The token cache is being invalidate because project 5afbe7e3fc474c1b9519be102c42d011 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:12.077293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:12.077925 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-cbab9a2f-cc38-43cf-beb8-07184024d9d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:12.079424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1248/2499] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:11 2022] DELETE /identity/v3/projects/5afbe7e3fc474c1b9519be102c42d011 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:12.084840 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.085339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.085459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.105286 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:12.106506 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:12.164578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1252/2500] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:12.174059 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.174281 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.174586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.194111 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:12.195328 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:12.204095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1249/2501] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:12.208246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.217810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1f5764-1230-4353-9091-374fa1dea173 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.222318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.222951 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.223233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.223509 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.224145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:12.224448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.226041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.237346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1253/2502] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:25:12.241570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.251254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3d9eeb-024c-4a97-8cdb-31b1805adb7b None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.255851 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.256433 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.256710 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.256980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.257611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.259556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.286706 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1250/2503] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:12.290660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.300687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ac101f-eba5-4d0a-a664-9234ddae75f6 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.327717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:12.361192 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.361955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.362234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.362538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.363165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.365105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.387957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1254/2504] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/users => generated 290 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:12.392053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.402994 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa16828-9f72-4ac4-a397-52ad6d32e12e admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.407666 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.408430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.408908 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.409387 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.410579 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.413618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.426731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1251/2505] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:12.430672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.440430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502c0302-dfdd-4435-b982-ba59b8d83944 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.445016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.445606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.445886 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.446161 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.446826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.448719 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c33c48d-1fb5-42c1-bc82-b0e230658b41 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.459031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1255/2506] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:25:12.463053 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.473316 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1c69a-6d91-49c4-a90c-44b60a13068a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.477924 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.478607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.478894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.479893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] PATH_INFO: `/v3/projects/dc9281a473c242bb904441846ce40f51/users/245c3499329c4c15a25f8b43f6a7161e/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.494495 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc9281a473c242bb904441846ce40f51, user_id=245c3499329c4c15a25f8b43f6a7161e, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.497203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.509941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1252/2507] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:12 2022] PUT /identity/v3/projects/dc9281a473c242bb904441846ce40f51/users/245c3499329c4c15a25f8b43f6a7161e/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:12.521819 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.522109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.522459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.552096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] MFA Rules not processed for user `245c3499329c4c15a25f8b43f6a7161e`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:12.553365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:12.615407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1256/2508] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:12.620275 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.681089 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e512e24-5d30-4ac2-b18b-20799b117c31 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.685745 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.686490 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.686644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.686918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.687544 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.689425 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.716400 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1253/2509] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/projects => generated 379 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:12.720627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.731352 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edf66b-6367-40cb-bed3-402a943d65e2 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.787057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.787660 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.787942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.788214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.788850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.790790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.815889 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1257/2510] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:12 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:12.817110 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.829581 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f37fc9-5230-42e5-a2df-77607dda5baa admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.834234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.834863 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.835144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.835413 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.836039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.837709 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.852873 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1254/2511] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:25:12.857258 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.867492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa9f6b2-a175-401e-857f-f3896b8ed094 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.872297 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.872945 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.873231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.873510 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] PATH_INFO: `/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6/users/4a57a0ec1a4a4c2baab34aecba5fee3b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.889061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c4d32c2cb64c25bde84e0cbad191f6, user_id=4a57a0ec1a4a4c2baab34aecba5fee3b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.891759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.904825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1258/2512] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:12 2022] PUT /identity/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6/users/4a57a0ec1a4a4c2baab34aecba5fee3b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:12.908988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:12.918818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd359d72-1240-43b0-a5e7-1d4198754218 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:12.938556 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:12.976431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:12.977032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:12.977313 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:12.977614 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:12.978242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:12.980079 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be142fc-4a43-4e46-bce6-59fb55ae94e4 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:12.989498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1255/2513] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:12 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:12.994096 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.000203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:13.006420 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e374d-79b9-4989-9048-dd0bd7bdc6fe admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.011981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.012614 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.012876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.013149 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] PATH_INFO: `/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6/users/4a57a0ec1a4a4c2baab34aecba5fee3b/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.015074 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=75c4d32c2cb64c25bde84e0cbad191f6, user_id=4a57a0ec1a4a4c2baab34aecba5fee3b, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.017669 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.027590 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1259/2514] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:12 2022] PUT /identity/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6/users/4a57a0ec1a4a4c2baab34aecba5fee3b/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:13.037569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.038031 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.038155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.065973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] MFA Rules not processed for user `4a57a0ec1a4a4c2baab34aecba5fee3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:13.067213 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:13.132439 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1256/2515] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:13.145241 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.147084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:13.204544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d384536-5a27-4737-a4b5-a1dbc9e7f84a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.208042 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.208634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.208911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.209181 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.218142 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.219588 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.228630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1260/2516] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:13.535254 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.544997 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e419b846-ddcf-470a-a493-61ad443633ac None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.550150 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a57a0ec1a4a4c2baab34aecba5fee3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75c4d32c2cb64c25bde84e0cbad191f6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.550816 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.551090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.551369 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.552016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:13.552325 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.553928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.565969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1257/2517] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:25:13.569965 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.579070 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172a4b1-9ea7-4acb-b739-290f8119cf36 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.583613 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a57a0ec1a4a4c2baab34aecba5fee3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75c4d32c2cb64c25bde84e0cbad191f6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.584204 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.584488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.584771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.585395 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.587329 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.613229 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1261/2518] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:13 2022] POST /identity/v3/projects => generated 354 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:13.617344 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.627178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b32658b-9ec5-4e52-98f0-5eaa1a055c30 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.682903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a57a0ec1a4a4c2baab34aecba5fee3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75c4d32c2cb64c25bde84e0cbad191f6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.683523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.684284 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.684750 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.685840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:13.685840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.687234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.700448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1258/2519] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:25:13.738132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:13.749105 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f4db41-79ba-4ca1-9ea9-a2f9eee79393 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:13.753672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4a57a0ec1a4a4c2baab34aecba5fee3b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '75c4d32c2cb64c25bde84e0cbad191f6', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:13.754295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:13.754606 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:13.754878 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] PATH_INFO: `/v3/projects/fbb6020bcdd74d97a79deb71800978a9` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:13.755989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorizing `identity:delete_project(project_id=fbb6020bcdd74d97a79deb71800978a9)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:13.757870 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:13.805168 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fbb6020bcdd74d97a79deb71800978a9', 'request_id': 'req-4017b202-5054-4c02-8257-b5b857cf2ba7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:13.812376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fbb6020bcdd74d97a79deb71800978a9', 'request_id': 'req-4017b202-5054-4c02-8257-b5b857cf2ba7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:13.817928 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fbb6020bcdd74d97a79deb71800978a9', 'request_id': 'req-4017b202-5054-4c02-8257-b5b857cf2ba7'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:13.829927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] The token cache is being invalidate because project fbb6020bcdd74d97a79deb71800978a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:13.830234 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:13.830880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-4017b202-5054-4c02-8257-b5b857cf2ba7 tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:13.832321 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1262/2520] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:13 2022] DELETE /identity/v3/projects/fbb6020bcdd74d97a79deb71800978a9 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:14.021568 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:14.023307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:14.053318 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=100 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:14.087412 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b908a231-dbef-4b6e-835c-26dce7a261be tempest-VolumeQuotaClassesTest-157659973 tempest-VolumeQuotaClassesTest-157659973-project-admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:14.142071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:14.142726 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:14.143013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:14.143289 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] PATH_INFO: `/v3/users/245c3499329c4c15a25f8b43f6a7161e` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:14.144421 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] RBAC: Authorizing `identity:delete_user(user_id=245c3499329c4c15a25f8b43f6a7161e)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:14.146317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:14.191459 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '245c3499329c4c15a25f8b43f6a7161e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.198288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '245c3499329c4c15a25f8b43f6a7161e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.198985 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '245c3499329c4c15a25f8b43f6a7161e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.206382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '245c3499329c4c15a25f8b43f6a7161e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.207039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '245c3499329c4c15a25f8b43f6a7161e'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.223809 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1259/2521] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:14 2022] DELETE /identity/v3/users/245c3499329c4c15a25f8b43f6a7161e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:14.228324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:14.228825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:14.228939 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:14.248595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:14.249794 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:14.295177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:14.310602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1263/2522] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:14.320979 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:14.322391 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:14.379503 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35debfde-f4dc-477d-8516-a263bd506602 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:14.383242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:14.383840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:14.384121 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:14.384398 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:14.392296 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:14.395030 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:14.396954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:14.409934 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1260/2523] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:14.693900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:14.695741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:14.709523 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36291154-76a5-4e5f-b22d-5d4e342a674f None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:14.715084 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:14.716381 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:14.716668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:14.717362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] PATH_INFO: `/v3/projects/dc9281a473c242bb904441846ce40f51` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:14.718156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc9281a473c242bb904441846ce40f51)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:14.720500 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:14.768679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc9281a473c242bb904441846ce40f51', 'request_id': 'req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.774850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc9281a473c242bb904441846ce40f51', 'request_id': 'req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.781406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc9281a473c242bb904441846ce40f51', 'request_id': 'req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.794376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] The token cache is being invalidate because project dc9281a473c242bb904441846ce40f51 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:14.794710 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.795350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.796857 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1264/2524] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:14 2022] DELETE /identity/v3/projects/dc9281a473c242bb904441846ce40f51 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:14.801324 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:14.811310 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e481397c-e5fc-4261-b84b-61e1f9b4ae6f service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:14.868955 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:14.869527 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:14.869806 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:14.870205 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] PATH_INFO: `/v3/users/4a57a0ec1a4a4c2baab34aecba5fee3b` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:14.871359 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a57a0ec1a4a4c2baab34aecba5fee3b)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:14.873893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:14.920848 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a57a0ec1a4a4c2baab34aecba5fee3b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.928101 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a57a0ec1a4a4c2baab34aecba5fee3b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.928991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a57a0ec1a4a4c2baab34aecba5fee3b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.937930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a57a0ec1a4a4c2baab34aecba5fee3b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.938759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-4608bd79-3c0f-42e2-b8e2-ec2e79d9f279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a57a0ec1a4a4c2baab34aecba5fee3b'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:14.961542 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1261/2525] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:14 2022] DELETE /identity/v3/users/4a57a0ec1a4a4c2baab34aecba5fee3b => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:15.246169 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.248741 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:15.262067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd28ee1d-90bc-4912-8fcf-aecf2108c48a admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.319544 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.320087 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.320364 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.320632 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] PATH_INFO: `/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.321720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75c4d32c2cb64c25bde84e0cbad191f6)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.323630 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.370061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75c4d32c2cb64c25bde84e0cbad191f6', 'request_id': 'req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:15.376328 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75c4d32c2cb64c25bde84e0cbad191f6', 'request_id': 'req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:15.381914 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75c4d32c2cb64c25bde84e0cbad191f6', 'request_id': 'req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:15.393942 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] The token cache is being invalidate because project 75c4d32c2cb64c25bde84e0cbad191f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:15.394231 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:15.394868 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ed81e2b1-a46d-4dab-9f0f-6738d6ba1816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:15.396324 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1265/2526] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:15 2022] DELETE /identity/v3/projects/75c4d32c2cb64c25bde84e0cbad191f6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:15.401526 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.401982 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.402081 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.424417 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:15.425602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:15.484457 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1262/2527] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:15.493053 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.493344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.493709 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.508518 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:15.515812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:15.517574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:15.526140 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1266/2528] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:15.531704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.541105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b87a62-bd5f-42f3-8741-b88bc8837077 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.546009 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.546627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.546906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.547174 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.547818 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:15.548106 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.549930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.561618 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1263/2529] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:25:15.565761 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.575333 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d6f5aa-c4d5-415a-9afb-39f3b1c576df None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.579876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.580459 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.580735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.581005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.581628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.583535 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.609994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1267/2530] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/projects => generated 399 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:15.614072 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.619061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:15.625159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85abdc46-b95b-4d76-805a-b6847e378558 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.682233 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.682868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.683149 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.683424 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.684057 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.687144 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.707940 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1264/2531] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:15.712147 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.722299 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0d267b-574c-4c65-be11-c660d78c2d12 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.727047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.727640 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.727919 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.728191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.728815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.730516 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.740751 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1268/2532] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:15.744561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.754493 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6eccca-1a90-4cd7-8a6c-7b4b8a0b2e3b admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.759035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.759619 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.759895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.760167 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.760800 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.763021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff22c2e-f7d9-4653-a687-1c31107263b3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.773212 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1265/2533] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Oct 23 02:25:15.777374 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.788014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f858332-8e16-48a9-aa67-2c3ec96fe049 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.792628 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.793267 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.793573 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.793855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] PATH_INFO: `/v3/projects/f1661030d2a64796a13d7322a76ec201/users/bc80de4f1fe4423f87e25fe40fa66c65/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.808522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1661030d2a64796a13d7322a76ec201, user_id=bc80de4f1fe4423f87e25fe40fa66c65, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:15.811237 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:15.824376 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1269/2534] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:15 2022] PUT /identity/v3/projects/f1661030d2a64796a13d7322a76ec201/users/bc80de4f1fe4423f87e25fe40fa66c65/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:15.835444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.835670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.835944 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.863027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] MFA Rules not processed for user `bc80de4f1fe4423f87e25fe40fa66c65`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:15.864217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:15.927055 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1266/2535] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:25:15.931497 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:15.992809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df02296-e979-4f77-8b7e-37a5a261f43f admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:15.998081 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:15.998708 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:15.998985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:15.999255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:15.999891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.001771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.033670 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1270/2536] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:15 2022] POST /identity/v3/projects => generated 401 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:16.034929 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.046012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669d6715-1015-4eb8-85c2-319fb5f0cd3e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.102634 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.103160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.103465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.103736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.104355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.106306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.125191 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:16.135016 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1267/2537] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:16 2022] POST /identity/v3/users => generated 307 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:16.136755 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.147759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369ad846-7b4e-4870-937c-b28ddfb364ca admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.152377 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.152972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.153255 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.153524 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.154156 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.155797 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.166189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1271/2538] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:16.170604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.180894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b4df2-abcf-4a0b-a239-c4bfda708d8a admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.186114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.186787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.187071 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.187348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] PATH_INFO: `/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f/users/7d09014c268d4b56ad40f39db4b6525a/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.202896 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac0f31288114e4dafd7f5e29bc7e38f, user_id=7d09014c268d4b56ad40f39db4b6525a, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.205537 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.218028 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1268/2539] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:16 2022] PUT /identity/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f/users/7d09014c268d4b56ad40f39db4b6525a/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:16.221990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.232117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cb2220-1d6b-47a7-925e-54dbb1063135 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.287806 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.288485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.288672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.288943 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.289568 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.291188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f25771f-ab61-42cc-b3ab-417511c3cdb4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.301940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1272/2540] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:16.306593 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.317586 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edd465d-46ca-40cc-a6cb-5cc90739c903 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.322217 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.322903 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.323188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.323465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] PATH_INFO: `/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f/users/7d09014c268d4b56ad40f39db4b6525a/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.325753 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac0f31288114e4dafd7f5e29bc7e38f, user_id=7d09014c268d4b56ad40f39db4b6525a, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.328455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.338394 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1269/2541] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:16 2022] PUT /identity/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f/users/7d09014c268d4b56ad40f39db4b6525a/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:16.349739 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.350031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.350309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.375877 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:16.379009 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] MFA Rules not processed for user `7d09014c268d4b56ad40f39db4b6525a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:16.380892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:16.448651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1273/2542] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:16 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:25:16.459293 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.461156 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:16.518260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a2688f-883f-44e6-84b1-91b1bd0656f3 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.521991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.522620 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.522898 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.523166 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.532226 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.533644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.542742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1270/2543] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:16 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:16.619117 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:16.629325 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dca0c8-2467-458a-8fae-de32b40bf4cc None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:16.632849 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:16.633442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:16.633721 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:16.633995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:16.690355 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:16.691898 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca0d91-8378-4d85-aa2b-4abbd37025cc service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:16.701484 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1274/2544] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:16 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:24.002696 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:24.064258 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32272bf-1476-4899-8ee3-17e63ee8c3c9 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:24.068736 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:24.069345 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:24.069622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:24.069893 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] PATH_INFO: `/v3/users/bc80de4f1fe4423f87e25fe40fa66c65` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:24.071014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc80de4f1fe4423f87e25fe40fa66c65)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:24.072890 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:24.117900 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc80de4f1fe4423f87e25fe40fa66c65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.124782 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc80de4f1fe4423f87e25fe40fa66c65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.126024 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc80de4f1fe4423f87e25fe40fa66c65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.133813 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc80de4f1fe4423f87e25fe40fa66c65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.134674 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc80de4f1fe4423f87e25fe40fa66c65'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.139253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:24.154090 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1271/2545] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:24 2022] DELETE /identity/v3/users/bc80de4f1fe4423f87e25fe40fa66c65 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:24.158690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:24.159179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:24.159323 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:24.179024 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:24.180235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:24.238826 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1275/2546] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:24.248455 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:24.249836 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:24.307855 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef1201c-9ba3-4bf4-833e-bca60f434290 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:24.311514 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:24.312114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:24.312395 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:24.312671 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:24.322105 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:24.323559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:24.332776 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1272/2547] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:24.600184 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:24.601773 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:24.618392 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb61a86f-b1ff-4944-ae80-ab040b5177b6 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:24.623724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:24.624537 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:24.624827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:24.625158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] PATH_INFO: `/v3/projects/f1661030d2a64796a13d7322a76ec201` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:24.627619 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1661030d2a64796a13d7322a76ec201)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:24.629581 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:24.667527 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:24.683365 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1661030d2a64796a13d7322a76ec201', 'request_id': 'req-0be29712-7e3b-4e3b-8e49-d9848f0dab06'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.692932 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1661030d2a64796a13d7322a76ec201', 'request_id': 'req-0be29712-7e3b-4e3b-8e49-d9848f0dab06'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.700057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1661030d2a64796a13d7322a76ec201', 'request_id': 'req-0be29712-7e3b-4e3b-8e49-d9848f0dab06'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.713318 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] The token cache is being invalidate because project f1661030d2a64796a13d7322a76ec201 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:24.713648 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.714232 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.715807 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1276/2548] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:24 2022] DELETE /identity/v3/projects/f1661030d2a64796a13d7322a76ec201 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:24.720917 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:24.730561 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df7a64-0372-4c2d-99c7-034007be64dd service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:24.792795 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:24.793430 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:24.793705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:24.793976 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] PATH_INFO: `/v3/users/7d09014c268d4b56ad40f39db4b6525a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:24.795128 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d09014c268d4b56ad40f39db4b6525a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:24.798242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:24.844959 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d09014c268d4b56ad40f39db4b6525a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.851999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d09014c268d4b56ad40f39db4b6525a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.852647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d09014c268d4b56ad40f39db4b6525a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.859722 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d09014c268d4b56ad40f39db4b6525a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.860928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-165e68d6-8d50-4524-ae91-30e145105a8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d09014c268d4b56ad40f39db4b6525a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:24.878129 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1273/2549] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:24 2022] DELETE /identity/v3/users/7d09014c268d4b56ad40f39db4b6525a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:25.151133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.152652 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:25.167704 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be29712-7e3b-4e3b-8e49-d9848f0dab06 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.226245 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.226829 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.227114 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.227391 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] PATH_INFO: `/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.228526 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac0f31288114e4dafd7f5e29bc7e38f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.230427 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.275727 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac0f31288114e4dafd7f5e29bc7e38f', 'request_id': 'req-ace323f0-c322-41a2-a1d2-41c0e9065b8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:25.282336 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac0f31288114e4dafd7f5e29bc7e38f', 'request_id': 'req-ace323f0-c322-41a2-a1d2-41c0e9065b8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:25.287975 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac0f31288114e4dafd7f5e29bc7e38f', 'request_id': 'req-ace323f0-c322-41a2-a1d2-41c0e9065b8c'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:25.300880 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] The token cache is being invalidate because project aac0f31288114e4dafd7f5e29bc7e38f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:25.301421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:25.302045 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-ace323f0-c322-41a2-a1d2-41c0e9065b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:25.303543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1277/2550] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:25 2022] DELETE /identity/v3/projects/aac0f31288114e4dafd7f5e29bc7e38f => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:25.308393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.308857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.309014 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.322592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:25.329294 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:25.331244 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:25.389644 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1274/2551] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:25.397602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.397890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.398165 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.418100 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:25.419339 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:25.427674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1278/2552] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:25.431860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.441998 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b87d562-3968-455e-80f4-5ba202e0000e None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.446574 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.447170 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.447452 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.447724 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] PATH_INFO: `/v3/domains` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.448361 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81625) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:25.448669 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.450255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.462020 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1275/2553] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:25:25.466127 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.476094 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e2d73-41d0-4e0f-b33f-b949cf094093 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.480717 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.481249 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.481525 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.481793 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.482416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.484337 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.511890 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1279/2554] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:25.516323 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.526522 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a23a128-a247-41af-b589-f7ac71d9c1be admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.582739 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.583330 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.583610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.583880 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.584496 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.586367 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.607569 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1276/2555] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:25.612436 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.622676 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4fa1f-aaee-4433-8948-097db9184773 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.627565 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.628157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.628468 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.628744 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.629400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.631031 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.640720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1280/2556] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:25.645093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.654677 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20c667-4d42-48bb-a9c7-9e9a27372272 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.660311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.660892 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.661177 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.661448 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.662069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.663672 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2e82ec-1c4c-4262-bea0-26f96b97f7e7 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.674420 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1277/2557] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:25.678679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.689777 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863eea2-0388-41f1-b4b3-896e73adc2c4 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.694668 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.695348 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.695674 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.695955 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] PATH_INFO: `/v3/projects/7b324b75feb54a26a9b222379eb78fa8/users/ed92fff3d5894f728bb3cd69159ec858/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.705273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:25.712735 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b324b75feb54a26a9b222379eb78fa8, user_id=ed92fff3d5894f728bb3cd69159ec858, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.716268 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.729498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1281/2558] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:25 2022] PUT /identity/v3/projects/7b324b75feb54a26a9b222379eb78fa8/users/ed92fff3d5894f728bb3cd69159ec858/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:25.739610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.739840 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.740115 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.768056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] MFA Rules not processed for user `ed92fff3d5894f728bb3cd69159ec858`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:25.769248 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:25.836332 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1278/2559] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:25.840748 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.902776 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a76ee386-a817-4da7-97c0-2f58203440d2 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:25.908002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:25.908602 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:25.908882 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:25.909157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] PATH_INFO: `/v3/projects` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:25.909779 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:25.911703 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:25.941047 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1282/2560] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/projects => generated 367 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:25.942562 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:25.948255 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:25.955733 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689197a4-10fd-4393-931b-efaf1e5f176f None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.011549 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.012077 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.012355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.012627 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] PATH_INFO: `/v3/users` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.013241 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.015192 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.036651 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1279/2561] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:25 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:26.040816 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.051904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f102f21-f65d-48b2-9dbb-8f07e9e32781 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.056481 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.057069 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.057342 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.057609 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.058233 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.059844 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.071157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1283/2562] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:26.075161 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.085601 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9032576-b448-45f7-8923-6f83c315d033 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.090159 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.090842 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.091124 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.091428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] PATH_INFO: `/v3/projects/b38565831cbe443292493b8fee59d540/users/23c4fb88150f413e9c9adc0dc4e107fd/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.105691 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38565831cbe443292493b8fee59d540, user_id=23c4fb88150f413e9c9adc0dc4e107fd, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.108358 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.120611 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1280/2563] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:26 2022] PUT /identity/v3/projects/b38565831cbe443292493b8fee59d540/users/23c4fb88150f413e9c9adc0dc4e107fd/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:26.124335 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.135157 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf237b7-0e5d-4c9d-8e5e-061a24c7db77 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.190759 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.191398 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.191622 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.191891 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.192519 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.194097 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c29a0f-7864-4eb1-8d2b-b4823d3df961 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.203778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1284/2564] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:26.208860 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.219006 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da21f488-21cd-4f66-ae0a-427430f47131 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.224058 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.224695 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.224973 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.225243 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] PATH_INFO: `/v3/projects/b38565831cbe443292493b8fee59d540/users/23c4fb88150f413e9c9adc0dc4e107fd/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.227134 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38565831cbe443292493b8fee59d540, user_id=23c4fb88150f413e9c9adc0dc4e107fd, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.229832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.240277 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1281/2565] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:26 2022] PUT /identity/v3/projects/b38565831cbe443292493b8fee59d540/users/23c4fb88150f413e9c9adc0dc4e107fd/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:26.252270 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.252556 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.252825 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.281225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] MFA Rules not processed for user `23c4fb88150f413e9c9adc0dc4e107fd`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:26.282389 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:26.326690 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:26.350756 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1285/2566] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:26 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:26.362936 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.364720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:26.422095 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1005eb-4722-4443-b447-56df192cc779 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.426592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.427185 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.427465 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.427738 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.436477 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.437891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.448085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1282/2567] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:26.530575 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:26.540429 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2697d093-22f7-4d6d-9aa9-a9e38430e6b0 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:26.543957 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:26.544549 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:26.544827 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:26.545101 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:26.603061 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:26.604498 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:26.614002 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1286/2568] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:34.334013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:34.335444 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:34.391876 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:34.405942 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a99d31-dd94-4aa1-beae-2933f607a027 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:34.411551 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:34.412151 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:34.412431 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:34.412703 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:34.414411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:34.426340 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:34.427954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff97213-f563-423e-968d-5eeaecc692cd service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:34.437943 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1283/2569] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:35.535786 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:35.599390 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbeae27-43c4-47de-8b0a-772eaa757f3a service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:35.603956 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:35.604580 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:35.604862 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:35.605145 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] PATH_INFO: `/v3/users/ed92fff3d5894f728bb3cd69159ec858` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:35.606251 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed92fff3d5894f728bb3cd69159ec858)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:35.608188 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:35.653306 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed92fff3d5894f728bb3cd69159ec858'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:35.660162 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed92fff3d5894f728bb3cd69159ec858'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:35.672411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed92fff3d5894f728bb3cd69159ec858'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:35.673196 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed92fff3d5894f728bb3cd69159ec858'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:35.674023 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed92fff3d5894f728bb3cd69159ec858'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:35.688039 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:35.693940 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1287/2570] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:35 2022] DELETE /identity/v3/users/ed92fff3d5894f728bb3cd69159ec858 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:35.698927 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:35.699334 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:35.699462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:35.719368 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:35.720545 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:35.778198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1284/2571] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:35.787948 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:35.789428 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:35.846179 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac02ad-1527-4ca1-86fc-7f20391ee501 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:35.849824 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:35.850358 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:35.850671 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:35.850944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:35.859743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:35.861191 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:35.870411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1288/2572] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:36.139034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:36.140481 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:36.152891 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d318f04-dfda-4a8f-aed2-b56a773a04e0 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:36.157462 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:36.158093 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.158373 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.158683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] PATH_INFO: `/v3/projects/7b324b75feb54a26a9b222379eb78fa8` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.159835 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b324b75feb54a26a9b222379eb78fa8)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:36.162008 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:36.209187 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b324b75feb54a26a9b222379eb78fa8', 'request_id': 'req-e5036bec-3dc5-4bb7-b99d-b559ae869e23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.215479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b324b75feb54a26a9b222379eb78fa8', 'request_id': 'req-e5036bec-3dc5-4bb7-b99d-b559ae869e23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.220956 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b324b75feb54a26a9b222379eb78fa8', 'request_id': 'req-e5036bec-3dc5-4bb7-b99d-b559ae869e23'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.234411 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] The token cache is being invalidate because project 7b324b75feb54a26a9b222379eb78fa8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:36.234725 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.235353 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.236826 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1285/2573] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:36 2022] DELETE /identity/v3/projects/7b324b75feb54a26a9b222379eb78fa8 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:36.241409 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:36.250905 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6f25f-fd08-4b46-8259-81a5e7b65588 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:36.306055 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:36.306724 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.307011 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.307288 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] PATH_INFO: `/v3/users/23c4fb88150f413e9c9adc0dc4e107fd` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.308447 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c4fb88150f413e9c9adc0dc4e107fd)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:36.310362 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:36.359033 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c4fb88150f413e9c9adc0dc4e107fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.367734 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c4fb88150f413e9c9adc0dc4e107fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.375718 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c4fb88150f413e9c9adc0dc4e107fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.376418 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c4fb88150f413e9c9adc0dc4e107fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.377123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-96b96fd9-c700-44b5-89dd-d219cb06474a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c4fb88150f413e9c9adc0dc4e107fd'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.395164 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1289/2574] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:25:36 2022] DELETE /identity/v3/users/23c4fb88150f413e9c9adc0dc4e107fd => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:36.676570 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:36.679500 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:36.698729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5036bec-3dc5-4bb7-b99d-b559ae869e23 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:36.755810 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:36.756501 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.756704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.756980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] PATH_INFO: `/v3/projects/b38565831cbe443292493b8fee59d540` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.758091 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38565831cbe443292493b8fee59d540)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:36.760034 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:36.770999 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:36.808003 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38565831cbe443292493b8fee59d540', 'request_id': 'req-d8f516b4-51d6-49b5-850a-4ea604e64a83'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.814240 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38565831cbe443292493b8fee59d540', 'request_id': 'req-d8f516b4-51d6-49b5-850a-4ea604e64a83'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.819864 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38565831cbe443292493b8fee59d540', 'request_id': 'req-d8f516b4-51d6-49b5-850a-4ea604e64a83'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.831969 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] The token cache is being invalidate because project b38565831cbe443292493b8fee59d540 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:25:36.832276 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.832870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-d8f516b4-51d6-49b5-850a-4ea604e64a83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:25:36.834963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1286/2575] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:25:36 2022] DELETE /identity/v3/projects/b38565831cbe443292493b8fee59d540 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:36.840200 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.840687 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.840809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.860361 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:36.861533 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:36.878138 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:36.923222 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1290/2576] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:36.931062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.931360 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.931641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.951307 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:36.952491 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:36.961154 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1287/2577] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:36.965732 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:36.975990 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae45b318-0a4b-4317-8e2c-1039d7095c61 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:36.980746 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:36.981344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:36.981625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:36.981900 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:36.982576 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:25:36.982884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:36.984485 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:36.996123 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1291/2578] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:25:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Oct 23 02:25:37.000414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.009911 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee6e68-4a9e-4a79-ac22-c4ebda13aa37 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.014592 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.015203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.015486 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.015759 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.016428 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.018351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.047741 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1288/2579] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:36 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:37.051911 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.062411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2815a0-ccbd-4142-93f8-c4e9017232a7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.120040 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.120726 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.121014 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.121296 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.122109 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.124108 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.146899 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1292/2580] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:37 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:37.150812 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.161062 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a3f11b-dd6d-4af7-95f8-344ddad11176 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.166434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.167070 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.167348 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.167756 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.168403 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.169991 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.183918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1289/2581] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:37.185057 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.195728 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a780015-b178-4046-9592-d36ae2aef4d7 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.200426 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.201121 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.201470 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.201812 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.202553 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.204223 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e174e892-31e2-44bc-ae02-3b2be2ab897b admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.215238 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1293/2582] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:37.219624 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.230032 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aa003a-68c3-497a-b05f-404ac3ad1b95 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.235015 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.235683 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.235941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.236215 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] PATH_INFO: `/v3/projects/d8d61c19477043d5bcb52018e0c38001/users/63fafcb4353e41fa87007c29d01aa041/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.251178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d61c19477043d5bcb52018e0c38001, user_id=63fafcb4353e41fa87007c29d01aa041, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.253837 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.267023 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1290/2583] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:37 2022] PUT /identity/v3/projects/d8d61c19477043d5bcb52018e0c38001/users/63fafcb4353e41fa87007c29d01aa041/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:37.272235 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34efec-13a9-487f-b207-17e54f96d1c5 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.272595 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34efec-13a9-487f-b207-17e54f96d1c5 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.272937 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce34efec-13a9-487f-b207-17e54f96d1c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.292482 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-ce34efec-13a9-487f-b207-17e54f96d1c5 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:37.293679 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ce34efec-13a9-487f-b207-17e54f96d1c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:37.352607 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1294/2584] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Oct 23 02:25:37.362253 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:37.365382 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:37.421705 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59febc-dec6-4a0a-b2eb-75446fab0b93 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:37.425357 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:37.425958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:37.426242 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:37.426566 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:37.436954 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:37.438319 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:37.448610 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1291/2585] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:38.109133 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.109651 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.109778 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.138421 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] MFA Rules not processed for user `63fafcb4353e41fa87007c29d01aa041`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:38.139440 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:38.158406 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:38.213048 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1295/2586] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:38 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:38.219223 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.229073 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b27e1a-b10d-4a42-b7cb-1a565ae6a820 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.233608 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.234202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.234506 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.234778 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.235415 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.237306 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.267456 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1292/2587] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:25:38 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:25:38.272214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.282981 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a832c-a840-4318-abb9-2ff08492cf0a None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.340923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.341532 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.341810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.342078 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.342730 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.344608 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.365557 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1296/2588] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:25:38 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Oct 23 02:25:38.369688 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.380256 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e18a95f-25da-4cb2-9241-28014856f208 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.385559 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.386160 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.386464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.386752 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.387393 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.389000 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.399539 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1293/2589] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:38.403578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.413944 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cda8e2d-75d2-4259-b097-c9b4d042a4a3 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.418493 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.419129 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.419407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.419681 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] PATH_INFO: `/v3/projects/7d34a00cced540c292352e2f3db54b91/users/98a7bf1f646a41e096e13dbd9cf85ff0/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.435225 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d34a00cced540c292352e2f3db54b91, user_id=98a7bf1f646a41e096e13dbd9cf85ff0, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.437845 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.450644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1297/2590] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:38 2022] PUT /identity/v3/projects/7d34a00cced540c292352e2f3db54b91/users/98a7bf1f646a41e096e13dbd9cf85ff0/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:25:38.454498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.464866 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbda07a1-8e38-4ca4-ba45-3eabfef87aa8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.468489 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:38.525039 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.525585 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.525853 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.526126 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.526796 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.528386 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832d5b22-2be7-43f2-8a34-ef1484681a78 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.537930 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1294/2591] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:25:38.542132 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:38.552340 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ca7b2-b7a5-4ceb-8043-fe9c29fa1938 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:38.557938 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:38.558627 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] REQUEST_METHOD: `PUT` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:38.558904 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:38.559177 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] PATH_INFO: `/v3/projects/7d34a00cced540c292352e2f3db54b91/users/98a7bf1f646a41e096e13dbd9cf85ff0/roles/88ada4935bda41c4bc55eea9219d9fe8` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:38.561076 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d34a00cced540c292352e2f3db54b91, user_id=98a7bf1f646a41e096e13dbd9cf85ff0, role_id=88ada4935bda41c4bc55eea9219d9fe8)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:38.563743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:38.575933 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1298/2592] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:25:38 2022] PUT /identity/v3/projects/7d34a00cced540c292352e2f3db54b91/users/98a7bf1f646a41e096e13dbd9cf85ff0/roles/88ada4935bda41c4bc55eea9219d9fe8 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:25:39.217767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:39.218239 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:39.218622 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:39.247909 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] MFA Rules not processed for user `98a7bf1f646a41e096e13dbd9cf85ff0`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:39.249204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:39.321189 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1295/2593] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:39.332762 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:39.335888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:39.398892 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82fbc53-33fc-4448-992a-b5c9bf8cf552 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:39.402455 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:39.403067 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:39.403344 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:39.403615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:39.429725 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:39.463439 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:39.464860 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ceb6fe-7384-40aa-b4aa-592137022d9d service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:39.473771 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1299/2594] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:40.853355 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:40.854798 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:40.867221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89a9e78-44f5-4513-a0b0-32833d570126 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:40.871043 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:40.871637 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:40.871918 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:40.872193 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:40.881921 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:40.883376 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:40.892598 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1296/2595] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:25:49.469743 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=16 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:50.888440 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=66 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:25:54.586202 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:54.587218 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:54.587347 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:54.607753 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] MFA Rules not processed for user `4f7ae78b38be4dee9ee5b695b05c2e3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:25:54.608923 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:54.663667 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1300/2596] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:25:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:25:54.900356 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:25:54.901760 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:54.960763 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a48eadd-8b16-4773-9899-d5b350b6af78 service cinder] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:25:54.965069 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:25:54.965673 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:25:54.965957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:25:54.966234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:25:54.967966 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:25:54.978007 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:25:54.979572 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:25:54.988513 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1297/2597] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:25:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:26:00.307597 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:00.373713 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63b9e3-9e22-439e-95c2-a77e0e33d4ef None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:00.378214 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:00.378888 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:00.379170 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:00.379445 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] PATH_INFO: `/v3/users/63fafcb4353e41fa87007c29d01aa041` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:00.380559 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=63fafcb4353e41fa87007c29d01aa041)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:00.382494 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:00.435159 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63fafcb4353e41fa87007c29d01aa041'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.443978 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63fafcb4353e41fa87007c29d01aa041'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.453018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63fafcb4353e41fa87007c29d01aa041'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.453922 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63fafcb4353e41fa87007c29d01aa041'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.454562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63fafcb4353e41fa87007c29d01aa041'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.477005 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1301/2598] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:26:00 2022] DELETE /identity/v3/users/63fafcb4353e41fa87007c29d01aa041 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:26:00.678027 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:00.695300 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee35e2-90d8-40bf-a5f5-467c2e8bc2a6 service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:00.754183 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:00.754868 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:00.755147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:00.755423 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] PATH_INFO: `/v3/projects/d8d61c19477043d5bcb52018e0c38001` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:00.756557 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d61c19477043d5bcb52018e0c38001)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:00.758473 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:00.806184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d61c19477043d5bcb52018e0c38001', 'request_id': 'req-eb065a03-8e38-42a8-a5d5-7d1055fe8097'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.816416 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d61c19477043d5bcb52018e0c38001', 'request_id': 'req-eb065a03-8e38-42a8-a5d5-7d1055fe8097'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.824670 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d61c19477043d5bcb52018e0c38001', 'request_id': 'req-eb065a03-8e38-42a8-a5d5-7d1055fe8097'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.839061 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] The token cache is being invalidate because project d8d61c19477043d5bcb52018e0c38001 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:26:00.839479 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.840188 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.841977 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1298/2599] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:26:00 2022] DELETE /identity/v3/projects/d8d61c19477043d5bcb52018e0c38001 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:26:00.847020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:00.848555 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:00.862884 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b268419e-0d92-46f8-bec1-0cfa2e188c73 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:00.889823 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:00.924810 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:00.925435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:00.925711 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:00.925979 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] PATH_INFO: `/v3/users/98a7bf1f646a41e096e13dbd9cf85ff0` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:00.927099 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98a7bf1f646a41e096e13dbd9cf85ff0)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:00.928994 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:00.977927 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98a7bf1f646a41e096e13dbd9cf85ff0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.985885 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98a7bf1f646a41e096e13dbd9cf85ff0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.993758 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98a7bf1f646a41e096e13dbd9cf85ff0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.995158 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98a7bf1f646a41e096e13dbd9cf85ff0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:00.995781 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-6c7d3b38-bded-44a1-a957-10f599694014 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98a7bf1f646a41e096e13dbd9cf85ff0'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.014407 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1302/2600] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:26:00 2022] DELETE /identity/v3/users/98a7bf1f646a41e096e13dbd9cf85ff0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:26:01.190155 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.191747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:01.205636 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb065a03-8e38-42a8-a5d5-7d1055fe8097 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.238186 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:01.267993 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.268628 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.268906 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.269202 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] PATH_INFO: `/v3/projects/7d34a00cced540c292352e2f3db54b91` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.270329 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d34a00cced540c292352e2f3db54b91)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.272267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.324013 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d34a00cced540c292352e2f3db54b91', 'request_id': 'req-bc36c004-a1b8-4066-9d71-b952ce50bed2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.330788 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d34a00cced540c292352e2f3db54b91', 'request_id': 'req-bc36c004-a1b8-4066-9d71-b952ce50bed2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.338641 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d34a00cced540c292352e2f3db54b91', 'request_id': 'req-bc36c004-a1b8-4066-9d71-b952ce50bed2'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.351922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] The token cache is being invalidate because project 7d34a00cced540c292352e2f3db54b91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:26:01.352221 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.352845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-bc36c004-a1b8-4066-9d71-b952ce50bed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:01.354532 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1299/2601] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:26:01 2022] DELETE /identity/v3/projects/7d34a00cced540c292352e2f3db54b91 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:26:01.360246 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.360795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.360795 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.380798 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:26:01.382000 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:01.441970 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1303/2602] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:26:01.451414 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.451654 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.451922 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.471538 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:26:01.472762 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:01.480957 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1300/2603] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:26:01.485492 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.495754 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e703d6b8-d9b3-4490-ab1e-5076f9fb7c10 None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.500572 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.501295 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.501644 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.501915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.502615 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81624) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Oct 23 02:26:01.502972 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.504564 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.517988 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1304/2604] 10.222.0.23 () {58 vars in 1153 bytes} [Sun Oct 23 02:26:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Oct 23 02:26:01.523645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.533267 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e541efb-9f70-4b16-bd44-0a33a7782d39 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.538036 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.538704 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.538988 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.539262 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.539895 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.541928 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.575832 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1301/2605] 10.222.0.23 () {60 vars in 1153 bytes} [Sun Oct 23 02:26:01 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:26:01.576995 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.587611 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f8a208-ecd2-4e23-b8aa-4e6333791f89 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.618435 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:01.651720 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.652384 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.652656 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.652929 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] PATH_INFO: `/v3/users` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.653586 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.655618 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.680789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1305/2606] 10.222.0.23 () {60 vars in 1144 bytes} [Sun Oct 23 02:26:01 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Oct 23 02:26:01.681958 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.694681 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cac0e6-ac41-45a8-abf3-4350cf2992b8 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.700147 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.700757 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.701044 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.701322 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.702021 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.703647 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.717720 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1302/2607] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:26:01.718984 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.728815 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110ef88a-38bb-4f3a-ae3a-7d5d50e4b9e5 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.733562 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.734203 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.734512 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.734789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.735430 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.737037 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a601b9-e8d9-48b7-b4a0-5b6afddb9d63 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.747578 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1306/2608] 10.222.0.23 () {58 vars in 1122 bytes} [Sun Oct 23 02:26:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Oct 23 02:26:01.751767 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.762820 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221af7-15e9-4e46-8295-1640600ecd05 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:01.767857 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:01.768203 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] REQUEST_METHOD: `PUT` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.768484 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.768844 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] PATH_INFO: `/v3/projects/76f58d27d22547cfa491f4a7777c15e0/users/775f15c2edfb474998b4942c51cc9101/roles/02c90212ffbd4aef9f8be3677f26ec3f` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.784408 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76f58d27d22547cfa491f4a7777c15e0, user_id=775f15c2edfb474998b4942c51cc9101, role_id=02c90212ffbd4aef9f8be3677f26ec3f)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:01.787107 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:01.802686 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1303/2609] 10.222.0.23 () {60 vars in 1483 bytes} [Sun Oct 23 02:26:01 2022] PUT /identity/v3/projects/76f58d27d22547cfa491f4a7777c15e0/users/775f15c2edfb474998b4942c51cc9101/roles/02c90212ffbd4aef9f8be3677f26ec3f => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:26:01.813625 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] REQUEST_METHOD: `POST` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:01.813915 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:01.814190 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:01.844558 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.auth.core [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] MFA Rules not processed for user `775f15c2edfb474998b4942c51cc9101`. Rule list: `[]` (Enabled: `True`). {{(pid=81624) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:26:01.845790 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:01.917397 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1307/2610] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:26:01 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:26:01.928470 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:01.930168 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:02.000894 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190fe88f-7fc5-45ce-856c-05c2874d8742 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:02.004564 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:02.005165 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:02.005475 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:02.005746 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:02.015337 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:02.016941 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d2915-f867-4df8-aa35-89362182f97a service cinder] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:02.027234 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1304/2611] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:26:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:26:11.493971 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:11.555985 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcf0a92-a3d2-4b8c-9861-eeaee68c450c None None] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:11.561574 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:11.562189 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:11.562488 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:11.562785 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] PATH_INFO: `/v3/users/775f15c2edfb474998b4942c51cc9101` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:11.563883 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=775f15c2edfb474998b4942c51cc9101)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:11.565789 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:11.609522 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:11.615634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775f15c2edfb474998b4942c51cc9101'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:11.624855 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775f15c2edfb474998b4942c51cc9101'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:11.634018 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775f15c2edfb474998b4942c51cc9101'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:11.635041 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775f15c2edfb474998b4942c51cc9101'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:11.636010 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775f15c2edfb474998b4942c51cc9101'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:11.655802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1308/2612] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:26:11 2022] DELETE /identity/v3/users/775f15c2edfb474998b4942c51cc9101 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:26:11.660436 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] REQUEST_METHOD: `POST` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:11.661012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:11.661012 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:11.668001 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:11.686652 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.auth.core [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] MFA Rules not processed for user `ef10648b88704f08acf4b18d8fe4cfac`. Rule list: `[]` (Enabled: `True`). {{(pid=81625) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Oct 23 02:26:11.687747 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:11.748729 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1305/2613] 10.222.0.23 () {58 vars in 958 bytes} [Sun Oct 23 02:26:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Oct 23 02:26:11.758350 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:11.759876 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:11.819598 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550c5b7b-0b52-4b70-bd9e-45e373e01f62 admin admin] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:11.823217 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:11.823820 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:11.824103 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:11.824382 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:11.833802 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:11.835273 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:11.844902 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1309/2614] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:26:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:26:12.104587 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:12.106056 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:12.121351 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245469-418e-4abb-b6fd-35accca29bc2 None None] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:12.127607 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:12.128317 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:12.128664 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:12.129035 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] PATH_INFO: `/v3/projects/76f58d27d22547cfa491f4a7777c15e0` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:12.130347 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76f58d27d22547cfa491f4a7777c15e0)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:12.132301 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:12.187498 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76f58d27d22547cfa491f4a7777c15e0', 'request_id': 'req-410c918f-7523-438c-8d63-ebde55143676'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:12.193742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76f58d27d22547cfa491f4a7777c15e0', 'request_id': 'req-410c918f-7523-438c-8d63-ebde55143676'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:12.199731 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76f58d27d22547cfa491f4a7777c15e0', 'request_id': 'req-410c918f-7523-438c-8d63-ebde55143676'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:12.212098 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] The token cache is being invalidate because project 76f58d27d22547cfa491f4a7777c15e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81625) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:26:12.212385 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:12.213017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:26:12.214523 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1306/2615] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:26:12 2022] DELETE /identity/v3/projects/76f58d27d22547cfa491f4a7777c15e0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:26:21.840401 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=40,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:22.209153 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:38.207400 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:38.208850 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:38.264449 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbc6273-1b0a-4231-8ccb-29557636e785 service neutron] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:38.311811 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a323cd646bae44b5a6f63714b5cdf70b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:38.312411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:38.312691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:38.312964 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:38.314697 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:38.384088 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:38.385442 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:38.396643 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1310/2616] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:26:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:26:40.990814 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:26:40.992288 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:41.048017 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c918f-7523-438c-8d63-ebde55143676 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:26:41.051596 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9900bcd989014ec783288c63d352d5c9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:26:41.052184 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:26:41.052604 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:26:41.053085 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:26:41.054602 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:26:41.068249 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:26:41.069797 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:26:41.080471 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1307/2617] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:26:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:26:48.391448 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=74 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:26:51.075770 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=26 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:27:09.280309 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:27:09.282699 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:27:09.341260 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:27:09.342787 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:27:09.360796 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef89e00d-3ad5-438a-b028-d04f08c47871 service cinder] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:27:09.365809 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:27:09.366411 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] REQUEST_METHOD: `GET` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:27:09.366745 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:27:09.367016 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:27:09.368752 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.fernet_utils [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81624) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:27:09.381197 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:27:09.382859 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:27:09.393604 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1311/2618] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:27:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Oct 23 02:27:09.411645 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3062d098-4ba6-4910-85e6-f4bb170a14fe service neutron] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:27:09.416206 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:27:09.416834 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:27:09.417114 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:27:09.417469 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] PATH_INFO: `/v3/users/4f7ae78b38be4dee9ee5b695b05c2e3a` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:27:09.418531 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f7ae78b38be4dee9ee5b695b05c2e3a)` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:27:09.420529 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:27:09.466434 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f7ae78b38be4dee9ee5b695b05c2e3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.473535 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f7ae78b38be4dee9ee5b695b05c2e3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.474204 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f7ae78b38be4dee9ee5b695b05c2e3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.481135 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f7ae78b38be4dee9ee5b695b05c2e3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.481870 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.notifications [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f7ae78b38be4dee9ee5b695b05c2e3a'} {{(pid=81625) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.498963 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1308/2619] 10.222.0.23 () {58 vars in 1224 bytes} [Sun Oct 23 02:27:09 2022] DELETE /identity/v3/users/4f7ae78b38be4dee9ee5b695b05c2e3a => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Oct 23 02:27:09.668279 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] Authenticating user token {{(pid=81624) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:27:09.682462 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10983b-59ea-4ee0-b28e-6d01668e27ad service placement] Validating token access rules against request {{(pid=81624) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:27:09.738346 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef10648b88704f08acf4b18d8fe4cfac', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ab10d627c6844be3824524caf5053de2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81624) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:27:09.738989 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:27:09.739266 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:27:09.739538 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] PATH_INFO: `/v3/projects/3f32b2691a2244adbaf9a37f6981b393` {{(pid=81624) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:27:09.740634 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f32b2691a2244adbaf9a37f6981b393)` {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:27:09.742543 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] RBAC: Authorization granted {{(pid=81624) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:27:09.787504 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f32b2691a2244adbaf9a37f6981b393', 'request_id': 'req-851ec72e-4673-4310-8c1d-1d1f19282129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.793672 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f32b2691a2244adbaf9a37f6981b393', 'request_id': 'req-851ec72e-4673-4310-8c1d-1d1f19282129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.799691 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f32b2691a2244adbaf9a37f6981b393', 'request_id': 'req-851ec72e-4673-4310-8c1d-1d1f19282129'} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.811404 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] The token cache is being invalidate because project 3f32b2691a2244adbaf9a37f6981b393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81624) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Oct 23 02:27:09.811694 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.813020 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG keystone.notifications [None req-851ec72e-4673-4310-8c1d-1d1f19282129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81624) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Oct 23 02:27:09.814686 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: [pid: 81624|app: 0|req: 1312/2620] 10.222.0.23 () {58 vars in 1233 bytes} [Sun Oct 23 02:27:09 2022] DELETE /identity/v3/projects/3f32b2691a2244adbaf9a37f6981b393 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Oct 23 02:27:19.495452 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:27:19.808416 dv-cinder-861491-5-iscsi devstack@keystone.service[81624]: DEBUG dbcounter [-] [81624] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81624) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Oct 23 02:30:36.785418 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Authenticating user token {{(pid=81625) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Oct 23 02:30:36.786845 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:30:36.924311 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe599550-d98f-4432-8fec-dbcba6768699 admin admin] Validating token access rules against request {{(pid=81625) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Oct 23 02:30:36.982643 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c845b9e9ea3f43bcbab59b306d6c5da0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '52677ee112d04bb4a55f42390446c4aa', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81625) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Oct 23 02:30:36.983178 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] REQUEST_METHOD: `GET` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Oct 23 02:30:36.983464 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] SCRIPT_NAME: `/identity` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Oct 23 02:30:36.983742 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81625) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Oct 23 02:30:36.985461 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.fernet_utils [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81625) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Oct 23 02:30:36.996621 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Oct 23 02:30:36.998198 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c97d78b-ff47-4e1a-88d4-69931a6e771a service placement] RBAC: Authorization granted {{(pid=81625) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Oct 23 02:30:37.007980 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: [pid: 81625|app: 0|req: 1309/2621] 10.222.0.23 () {60 vars in 1365 bytes} [Sun Oct 23 02:30:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Oct 23 02:30:47.003744 dv-cinder-861491-5-iscsi devstack@keystone.service[81625]: DEBUG dbcounter [-] [81625] Writing DB stats keystone:SELECT=89 {{(pid=81625) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}